RI: Small: Inverse Rendering by Co-Evolutionary Learning

RI：小：通过共同进化学习进行逆向渲染

• 批准号: 1617767
• 负责人: Jia Deng
• 金额: $ 45.07万
• 依托单位: Regents of the University of Michigan - Ann Arbor
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2016
• 资助国家: 美国
• 起止时间: 2016-06-15 至 2018-10-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1617767&HistoricalAwards=false
• 关键词: RI; Small; Inverse; Rendering; Co

This project addresses the problem of inverse rendering: recovering 3D shape, material, and lighting from a single image. Inverse rendering is a fundamental problem in computer vision; it recovers the basic properties of a visual scene, and serves as a foundation for higher-level scene understanding such as recognizing objects, actions, and functionalities. Despite its fundamental importance, inverse rendering remains difficult. Solving inverse rendering can significantly advance computer vision and benefit a wide variety of applications from autonomous driving to assisting the visually impaired. This project develops new machine learning algorithms to advance the state of the art of inverse rendering. In addition, the project contributes to education and diversity by integrating research results into courses at various levels and by recruiting underrepresented groups to participate in this research. This research advances inverse rendering technologies using computer graphics and machine learning. In particular, the research team develops two machine learning systems that co-evolve as adversaries: a rendering system that learns to compose 3D scenes and renders images using a graphics engine, and an inverse rendering system that learns to recover shape, material, and lighting from the rendered images. To develop the rendering system, the research team investigates new learning algorithms for adaptive, automatic scene composition. To develop the inverse rendering system, the research team investigates new learning algorithms that integrate neural networks and physics-based vision.

该项目解决了逆向渲染的问题：从单个图像恢复 3D 形状、材质和光照。逆向渲染是计算机视觉中的一个基本问题；它恢复了视觉场景的基本属性，并作为更高层次的场景理解（例如识别对象、动作和功能）的基础。尽管逆渲染具有根本重要性，但它仍然很困难。解决逆向渲染问题可以显着推进计算机视觉，并使从自动驾驶到帮助视障人士的各种应用受益。该项目开发新的机器学习算法，以推进逆渲染技术的发展。此外，该项目通过将研究成果整合到各级课程中并招募代表性不足的群体参与这项研究，为教育和多样性做出了贡献。这项研究利用计算机图形和机器学习推进逆渲染技术。特别是，研究团队开发了两个作为对手共同进化的机器学习系统：一个学习构建 3D 场景并使用图形引擎渲染图像的渲染系统，以及一个学习恢复形状、材质和照明的逆渲染系统从渲染的图像中。为了开发渲染系统，研究团队研究了用于自适应自动场景合成的新学习算法。为了开发逆渲染系统，研究团队研究了集成神经网络和基于物理的视觉的新学习算法。