CRII: RI: General Surface Reconstruction via Polarized Computational Imaging

CRII：RI：通过偏振计算成像进行一般表面重建

• 批准号: 1948524
• 负责人: Jinwei Ye
• 金额: $ 17.5万
• 依托单位: Louisiana State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-04-01 至 2023-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1948524&HistoricalAwards=false
• 关键词: CRII; RI; General; Surface; Reconstruction

Complete and faithful digital reconstruction of the real world is critical to many applications, such as virtual and augmented reality, holographic rendering, autonomous driving, digital cultural heritage preservation, virtual architecture and interior design. The physical world consists of objects made with different materials that give them diverse appearances. Scanning a scene with high visual diversity is challenging. All existing 3D scanning systems are limited to specific types of surfaces that they can work for. For example, some only works for the matte surfaces while some are designed for highly reflective ones. Therefore, there is need to devise a generic solution for digitalizing the real-world surfaces with diverse appearances. This project will result in an integrated solution for reconstructing general surfaces, regardless of their appearances. The outcome will significantly simplify the 3D scanning procedure and allow wide use of this technology by the general public. In addition, this research will be integrated into the curricula of new courses on computer vision and computational imaging at Louisiana State University.This project will develop robust, fast, and accurate polarization-based imaging system for reconstructing general surfaces, ranging from diffuse to purely specular. This research is motivated by the fact that the polarization state of light embeds essential information on light transport. To achieve the goal, this project has two research thrusts: 1) formulate a polarization-inclusive light transport framework and 2) develop a practical system for reconstructing general surfaces. The first thrust will characterize the transformation of polarization after light interacts with surfaces of various reflectance properties. This framework provides the theoretical foundation for system development. The second thrust will design new programmable polarization-coded light sources and cameras, by leveraging recent advances in polarization sensors and optics. New physics-based 3D surface reconstruction algorithms will be developed through polarization analysis. Result of this project is expected to breakthrough major limitations of existing 3D scanning technology on general applicability.applicability.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

完整、忠实地数字化重建现实世界对于许多应用至关重要，例如虚拟和增强现实、全息渲染、自动驾驶、数字文化遗产保护、虚拟建筑和室内设计。物理世界由不同材料制成的物体组成，这些材料赋予它们不同的外观。扫描具有高度视觉多样性的场景具有挑战性。所有现有的 3D 扫描系统都仅限于它们可以工作的特定类型的表面。例如，有些仅适用于无光泽表面，而有些则设计用于高反射表面。因此，需要设计一种通用的解决方案来数字化具有不同外观的现实世界表面。该项目将产生一个用于重建一般表面的集成解决方案，无论其外观如何。结果将显着简化 3D 扫描程序，并允许公众广泛使用该技术。此外，这项研究将被纳入路易斯安那州立大学计算机视觉和计算成像新课程的课程中。该项目将开发强大、快速、准确的基于偏振的成像系统，用于重建一般表面，从漫射到纯表面。镜面反射。这项研究的动机是光的偏振态嵌入了光传输的基本信息。为了实现这一目标，该项目有两个研究重点：1）制定包含偏振的光传输框架；2）开发用于重建一般表面的实用系统。第一个推力将表征光与各种反射特性的表面相互作用后偏振的转变。该框架为系统开发提供了理论基础。第二个重点是利用偏振传感器和光学器件的最新进展，设计新的可编程偏振编码光源和相机。新的基于物理的 3D 表面重建算法将通过偏振分析来开发。该项目的成果预计将突破现有 3D 扫描技术在一般适用性方面的主要限制。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Polarimetric Helmholtz Stereopsis

偏振亥姆霍兹立体观测

• DOI: 10.1109/iccv48922.2021.00499
• 发表时间: 2021
• 期刊: CVF/IEEE International Conference on Computer Vision
• 作者: Ding, Yuqi;Ji, Yu;Zhou, Mingyuan;Kang, Sing Bing;Ye, Jinwei
• 通讯作者: Ye, Jinwei

Learning to Remove Refractive Distortions from Underwater Images

• DOI: 10.1109/iccv48922.2021.00496
• 发表时间: 2021-10
• 期刊: 2021 IEEE/CVF International Conference on Computer Vision (ICCV)
• 作者: Simron Thapa;Nianyi Li;Jinwei Ye

Polar-Photometric Stereo Under Natural Illumination

自然光照下的极光立体

• DOI: 10.1109/3dv57658.2022.00079
• 发表时间: 2022
• 期刊: 2022 International Conference on 3D Vision (3DV
• 作者: Ding, Yuqi;Ji, Yu;Ye, Jinwei
• 通讯作者: Ye, Jinwei

Unsupervised Non-Rigid Image Distortion Removal via Grid Deformation

• DOI: 10.1109/iccv48922.2021.00252
• 发表时间: 2021-01-01
• 期刊: 2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021)
• 作者: Li, Nianyi;Thapa, Simron;Ye, Jinwei
• 通讯作者: Ye, Jinwei