CRII: RI: General Surface Reconstruction via Polarized Computational Imaging

CRII：RI：通过偏振计算成像进行一般表面重建

• 批准号: 1948524
• 负责人: Jinwei Ye
• 金额: $ 17.5万
• 依托单位: Louisiana State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-04-01 至 2023-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1948524&HistoricalAwards=false
• 关键词: CRII; RI; General; Surface; Reconstruction

Complete and faithful digital reconstruction of the real world is critical to many applications, such as virtual and augmented reality, holographic rendering, autonomous driving, digital cultural heritage preservation, virtual architecture and interior design. The physical world consists of objects made with different materials that give them diverse appearances. Scanning a scene with high visual diversity is challenging. All existing 3D scanning systems are limited to specific types of surfaces that they can work for. For example, some only works for the matte surfaces while some are designed for highly reflective ones. Therefore, there is need to devise a generic solution for digitalizing the real-world surfaces with diverse appearances. This project will result in an integrated solution for reconstructing general surfaces, regardless of their appearances. The outcome will significantly simplify the 3D scanning procedure and allow wide use of this technology by the general public. In addition, this research will be integrated into the curricula of new courses on computer vision and computational imaging at Louisiana State University.This project will develop robust, fast, and accurate polarization-based imaging system for reconstructing general surfaces, ranging from diffuse to purely specular. This research is motivated by the fact that the polarization state of light embeds essential information on light transport. To achieve the goal, this project has two research thrusts: 1) formulate a polarization-inclusive light transport framework and 2) develop a practical system for reconstructing general surfaces. The first thrust will characterize the transformation of polarization after light interacts with surfaces of various reflectance properties. This framework provides the theoretical foundation for system development. The second thrust will design new programmable polarization-coded light sources and cameras, by leveraging recent advances in polarization sensors and optics. New physics-based 3D surface reconstruction algorithms will be developed through polarization analysis. Result of this project is expected to breakthrough major limitations of existing 3D scanning technology on general applicability.applicability.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

对现实世界的完整而忠实的数字重建对于许多应用程序至关重要，例如虚拟和增强现实，全息渲染，自动驾驶，数字文化遗产保护，虚拟建筑和室内设计。物理世界由用不同材料制成的物体组成，这些物品使它们具有多样化的外观。扫描具有高视觉多样性的场景是具有挑战性的。所有现有的3D扫描系统都仅限于它们可以使用的特定表面。例如，有些仅适用于哑光表面，而有些则是为高度反射性的。因此，有必要设计一种通用解决方案，以数字化具有不同外观的现实表面。该项目将产生一个集成的解决方案，用于重建一般表面，无论其外观如何。该结果将大大简化3D扫描程序，并允许公众广泛使用该技术。此外，这项研究将集成到路易斯安那州立大学的计算机视觉和计算成像的新课程中。该项目将开发出可靠，快速且基于准确的基于极化成像系统，用于重建一般表面，从弥漫到弥漫到纯粹的镜头。这项研究是由于光的极化状态嵌入了光传输的基本信息。为了实现目标，该项目有两个研究作用：1）制定一个包括极化的光传输框架，2）开发一个实用系统来重建一般表面。第一个推力将表征光在光与各种反射特性的表面相互作用之后的变化。该框架为系统开发提供了理论基础。第二个推力将通过利用偏振传感器和光学的最新进展来设计新的可编程极化编码的光源和相机。新的基于物理学的3D表面重建算法将通过极化分析开发。预计该项目的结果将在一般适用性上突破现有的3D扫描技术的重大局限性。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的知识分子优点和更广泛的影响评估的评估评估来支持的。

项目成果

Polarimetric Helmholtz Stereopsis

偏振亥姆霍兹立体观测

• DOI: 10.1109/iccv48922.2021.00499
• 发表时间: 2021
• 期刊: CVF/IEEE International Conference on Computer Vision
• 作者: Ding, Yuqi;Ji, Yu;Zhou, Mingyuan;Kang, Sing Bing;Ye, Jinwei
• 通讯作者: Ye, Jinwei

Learning to Remove Refractive Distortions from Underwater Images

• DOI: 10.1109/iccv48922.2021.00496
• 发表时间: 2021-10
• 期刊: 2021 IEEE/CVF International Conference on Computer Vision (ICCV)
• 作者: Simron Thapa;Nianyi Li;Jinwei Ye

Polar-Photometric Stereo Under Natural Illumination

自然光照下的极光立体

• DOI: 10.1109/3dv57658.2022.00079
• 发表时间: 2022
• 期刊: 2022 International Conference on 3D Vision (3DV
• 作者: Ding, Yuqi;Ji, Yu;Ye, Jinwei
• 通讯作者: Ye, Jinwei

Unsupervised Non-Rigid Image Distortion Removal via Grid Deformation

• DOI: 10.1109/iccv48922.2021.00252
• 发表时间: 2021-01-01
• 期刊: 2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021)
• 作者: Li, Nianyi;Thapa, Simron;Ye, Jinwei
• 通讯作者: Ye, Jinwei