CAREER: Towards Polarimetric Visual Understanding

职业：走向偏振视觉理解

• 批准号: 2238141
• 负责人: Jinwei Ye
• 金额: $ 60万
• 依托单位: George Mason University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-07-01 至 2028-06-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2238141&HistoricalAwards=false
• 关键词: CAREER; Towards; Polarimetric; Visual; Understanding

Polarized light is very common in our surroundings. As most visual inference algorithms leverage the color and brightness of images, the use of polarization in computer vision has not yet reached its full potential. This CAREER project will study how the way surfaces reflect polarized light can be used to help recognize objects more effectively. The project will address the following two questions: How does the polarization of light change after interacting with various types of surfaces? What can polarized light tell us about the kinds of objects that it has interacted with? Addressing these questions can lead to significant improvements in machine vision systems by strengthening their capability to do geometric and semantic scene understanding in which shapes are described and named. This project will result in new polarization-based vision systems for autonomous navigation and smart manufacturing. It will also produce novel solutions to the challenging problems of imaging and sensing through scattering media, including water, fog, haze, clouds, and bodily tissues.This project will investigate polarimetric light transport and tackle the problem of polarimetric visual understanding through analysis by synthesis. The research team will derive theoretical models for polarimetric surface reflectance and volumetric light transport, design visual inference algorithms for scene understanding, and develop computational imaging systems for real-world data acquisition. Specifically, the project will first study the transfer of polarization state upon local surface interactions. Physics-based layered material models will be adopted for characterizing complex surface scattering effects beyond direct mirror reflection. The project will then study the transport of polarized light through a spatial volume. A bottom-up approach will be taken to develop a voxel-based light transport model to mitigate the ill-posed transport matrix decomposition problem. The project will also study the challenging scenarios of turbid media, which could result in new techniques for imaging through scattering media. Leveraging the polarimetric light transport models, visual inference algorithms will be designed and developed for understanding scene properties, such as depth, shape, reflectance, and semantic composition. This project will develop novel computational imaging systems for acquiring real-world data to validate and analyze our proposed models and algorithms. A high precision polarimetric reflectance of various surfaces will be collected in this project and shared with the research community. Other broader impacts of this project include integrating the research results into existing and new course curricula on computer vision.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

偏振光在我们的周围环境中很常见。由于大多数视觉推理算法都利用图像的颜色和亮度，因此计算机视觉中偏振的使用尚未充分发挥其潜力。该职业项目将研究表面反射偏振光的方式如何用于帮助更有效地识别物体。该项目将解决以下两个问题：光与各种类型的表面相互作用后，偏振如何变化？偏振光可以告诉我们有关它与之相互作用的物体类型的哪些信息？解决这些问题可以通过加强机器视觉系统进行几何和语义场景理解（其中描述和命名形状）的能力来显着改进机器视觉系统。该项目将为自主导航和智能制造带来新的基于偏振的视觉系统。它还将为通过散射介质（包括水、雾、霾、云和身体组织）成像和传感的挑战性问题提供新颖的解决方案。该项目将研究偏振光传输并通过综合分析解决偏振视觉理解问题。研究团队将推导出偏振表面反射率和体积光传输的理论模型，设计用于场景理解的视觉推理算法，并开发用于现实世界数据采集的计算成像系统。具体来说，该项目将首先研究局部表面相互作用下偏振态的转移。将采用基于物理的分层材料模型来表征直接镜面反射之外的复杂表面散射效应。该项目随后将研究偏振光在空间体积中的传输。将采用自下而上的方法来开发基于体素的光传输模型，以减轻不适定的传输矩阵分解问题。该项目还将研究浑浊介质的挑战性场景，这可能会产生通过散射介质成像的新技术。利用偏振光传输模型，将设计和开发视觉推理算法来理解场景属性，例如深度、形状、反射率和语义构成。该项目将开发新颖的计算成像系统，用于获取真实世界的数据，以验证和分析我们提出的模型和算法。该项目将收集各种表面的高精度偏振反射率，并与研究界共享。该项目的其他更广泛的影响包括将研究成果整合到现有和新的计算机视觉课程中。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。