Collaborative Research: CIF: Medium: Snapshot Computational Imaging with Metaoptics

合作研究：CIF：Medium：Metaoptics 快照计算成像

• 批准号: 2403123
• 负责人: Vishwanath Saragadam Raja Venkata
• 金额: $ 40万
• 依托单位: University of California-Riverside
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2024
• 资助国家: 美国
• 起止时间: 2024-07-01 至 2027-06-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2403123&HistoricalAwards=false
• 关键词: Collaborative; Research; CIF; Medium; Snapshot

Light interacts richly with materials in the world along its many axes, including spatial, temporal, angular, spectral, and polarization. Measuring these interactions at a fine-grained scale is a key enabling technology in numerous scientific endeavors, including life sciences, remote sensing, security and forensics, and augmented and virtual reality. Yet, traditional image sensors capture only two-dimensional spatial variations. Therefore, other properties of light are measured by either embedding them within the spatial dimensions (e.g., Bayer-color or polarization-filter mosaics spatially spread over the sensor) or capturing them sequentially (e.g., acquiring consecutive video frames in the time dimension or consecutive hyperspectral components in the spectral dimension). However, snapshot approaches that use spatial tiling, while simple, lead to aliasing artifacts and invariably require expensive manufacturing techniques (bonding color/polarization filters to the sensor array). On the other hand, sequential measurements entail motion artifacts and lower frame rates. In contrast, this project develops snapshot computational cameras for capturing information along light's various dimensions by leveraging recent advances in metaoptics, i.e., optical devices that use sub-wavelength nano-structures to manipulate light characteristics - such as phase, wavelength, amplitude, or polarization - with a degree of control not feasible in traditional refractive optics. The project focuses on developing metaoptics-based imaging systems with frequency-domain multiplexing instead of spatial tiling or sequential imaging. Such frequency-multiplexed techniques require minimal changes to existing imaging systems while enabling snapshot measurements of multiple dimensions with minimal aliasing artifacts. The project will focus on three main objectives to achieve snapshot computational-imaging systems. The first objective is to build simulators based on rendering algorithms for computational cameras, as well as combinations and differentiable versions of such cameras, to efficiently simulate the various dimensions of light, including time-of-flight, spectrum, and polarization. In tandem, a scalable, differentiable metaoptics simulator will be built that can handle wave effects as light interacts with the metaoptical nano-structures which are smaller than the wavelength of the light. The second objective is to design frequency-multiplexed snapshot cameras by leveraging the simulators developed in the first objective, leading to metaoptics-based cameras that enable capturing intensity over large depths of field with high numerical aperture, thereby achieving compact imaging systems with low operational power. The third objective is to demonstrate the advantages of snapshot cameras by designing and building lab prototypes and comparing them against current state-of-the-art imagers. The outcomes of this project will impact various disciplines, including computer graphics, optics, computational imaging, and biomedical imaging.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

光沿着其许多轴（包括空间、时间、角度、光谱和偏振）与世界上的物质发生丰富的相互作用。在细粒度范围内测量这些相互作用是众多科学事业中的一项关键支持技术，包括生命科学、遥感、安全和取证以及增强现实和虚拟现实。然而，传统的图像传感器只能捕获二维空间变化。因此，光的其他属性可以通过将它们嵌入空间维度（例如，拜耳颜色或偏振滤光片马赛克在空间上分布在传感器上）或顺序捕获它们（例如，获取时间维度中的连续视频帧或连续的视频帧）来测量。光谱维度中的高光谱分量）。然而，使用空间平铺的快照方法虽然简单，但会导致混叠伪影，并且总是需要昂贵的制造技术（将彩色/偏振滤光片粘合到传感器阵列）。另一方面，连续测量会带来运动伪影和较低的帧速率。相比之下，该项目开发了快照计算相机，利用元光学的最新进展，即使用亚波长纳米结构来操纵光特性（例如相位、波长、振幅或偏振）的光学设备，用于沿光的各个维度捕获信息- 具有传统折射光学中不可行的一定程度的控制。该项目的重点是开发基于元光学的成像系统，具有频域复用而不是空间平铺或顺序成像。这种频率复用技术需要对现有成像系统进行最小的改变，同时能够以最小的混叠伪影实现多个维度的快照测量。该项目将重点关注实现快照计算成像系统的三个主要目标。第一个目标是基于计算相机的渲染算法以及此类相机的组合和可微分版本构建模拟器，以有效地模拟光的各个维度，包括飞行时间、光谱和偏振。同时，将构建一个可扩展、可微分的超光学模拟器，该模拟器可以处理光与小于光波长的超光学纳米结构相互作用时的波效应。第二个目标是利用第一个目标中开发的模拟器来设计频率复用快照相机，从而产生基于超光学的相机，能够以高数值孔径捕获大景深的强度，从而实现低运行功率的紧凑成像系统。第三个目标是通过设计和构建实验室原型并将其与当前最先进的成像仪进行比较来展示快照相机的优势。该项目的成果将影响多个学科，包括计算机图形学、光学、计算成像和生物医学成像。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。