Collaborative Research: CIF: Medium: Snapshot Computational Imaging with Metaoptics

合作研究：CIF：Medium：Metaoptics 快照计算成像

• 批准号: 2403122
• 负责人: Adithya Pediredla
• 金额: $ 80万
• 依托单位: Dartmouth College
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2024
• 资助国家: 美国
• 起止时间: 2024-07-01 至 2027-06-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2403122&HistoricalAwards=false
• 关键词: Collaborative; Research; CIF; Medium; Snapshot

Light interacts richly with materials in the world along its many axes, including spatial, temporal, angular, spectral, and polarization. Measuring these interactions at a fine-grained scale is a key enabling technology in numerous scientific endeavors, including life sciences, remote sensing, security and forensics, and augmented and virtual reality. Yet, traditional image sensors capture only two-dimensional spatial variations. Therefore, other properties of light are measured by either embedding them within the spatial dimensions (e.g., Bayer-color or polarization-filter mosaics spatially spread over the sensor) or capturing them sequentially (e.g., acquiring consecutive video frames in the time dimension or consecutive hyperspectral components in the spectral dimension). However, snapshot approaches that use spatial tiling, while simple, lead to aliasing artifacts and invariably require expensive manufacturing techniques (bonding color/polarization filters to the sensor array). On the other hand, sequential measurements entail motion artifacts and lower frame rates. In contrast, this project develops snapshot computational cameras for capturing information along light's various dimensions by leveraging recent advances in metaoptics, i.e., optical devices that use sub-wavelength nano-structures to manipulate light characteristics - such as phase, wavelength, amplitude, or polarization - with a degree of control not feasible in traditional refractive optics. The project focuses on developing metaoptics-based imaging systems with frequency-domain multiplexing instead of spatial tiling or sequential imaging. Such frequency-multiplexed techniques require minimal changes to existing imaging systems while enabling snapshot measurements of multiple dimensions with minimal aliasing artifacts. The project will focus on three main objectives to achieve snapshot computational-imaging systems. The first objective is to build simulators based on rendering algorithms for computational cameras, as well as combinations and differentiable versions of such cameras, to efficiently simulate the various dimensions of light, including time-of-flight, spectrum, and polarization. In tandem, a scalable, differentiable metaoptics simulator will be built that can handle wave effects as light interacts with the metaoptical nano-structures which are smaller than the wavelength of the light. The second objective is to design frequency-multiplexed snapshot cameras by leveraging the simulators developed in the first objective, leading to metaoptics-based cameras that enable capturing intensity over large depths of field with high numerical aperture, thereby achieving compact imaging systems with low operational power. The third objective is to demonstrate the advantages of snapshot cameras by designing and building lab prototypes and comparing them against current state-of-the-art imagers. The outcomes of this project will impact various disciplines, including computer graphics, optics, computational imaging, and biomedical imaging.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

光与世界上许多轴的材料相互作用，包括空间，时间，角，光谱和极化。以细粒度测量这些相互作用是许多科学努力中的关键促进技术，包括生命科学，遥感，安全性和取证以及增强和虚拟现实。然而，传统图像传感器仅捕获二维空间变化。因此，光的其他特性是通过将它们嵌入空间维度（例如，拜耳或极化过滤器的镶嵌物在空间上分布在传感器上）或顺序捕获它们（例如，在时间维度或连续的视频帧中以时间维度或连续的视频镜框捕获）。但是，快照的方法使用空间瓷砖，尽管简单，但会导致伪像，并且总是需要昂贵的制造技术（将颜色/极化过滤器粘合到传感器阵列）。另一方面，顺序测量需要运动伪影和较低的帧速率。相比之下，该项目开发了快照计算相机，用于通过利用最新的元元技术进步来捕获Light的各个维度，即使用亚波长纳米结构来操纵光特性的光学设备，例如相位，波长，波长，振幅，振幅，或极化程度 - 具有传统的反映能力，在传统的反映方面不可行。该项目着重于开发具有频域多路复用的基于元词的成像系统，而不是空间瓷砖或顺序成像。这样的频率 - 培训技术需要对现有成像系统的最小变化，同时可以使用最小的伪影对多个维度进行快照测量。该项目将集中在三个主要目标上，以实现快照计算成像系统。第一个目的是基于计算摄像机的渲染算法以及此类相机的组合和可区分版本来构建模拟器，以有效地模拟光的各个尺寸，包括飞行时间，光谱和极化。同时，将构建一个可扩展的，可区分的元磁模拟器，该模拟器可以在光与光波长小的元自动纳米结构相互作用时处理波浪效应。第二个目标是通过利用在第一个目标中开发的模拟器来设计频率的快照摄像机，从而导致基于元词的摄像机，从而使具有高数值的孔径在较大的场地上捕获强度，从而实现具有低操作功率的紧凑型成像系统。第三个目标是通过设计和构建实验室原型来证明快照相机的优势，并将其与当前的最新成像器进行比较。该项目的结果将影响各种学科，包括计算机图形，光学，计算成像和生物医学成像。该奖项反映了NSF的法定任务，并被认为是通过基金会的知识分子和更广泛的影响评估标准的评估来通过评估来支持的。