Collaborative Research: OP: Meta-optical Computational Image Sensors

合作研究：OP：元光学计算图像传感器

• 批准号: 2127235
• 负责人: Arka Majumdar
• 金额: $ 27.5万
• 依托单位: University of Washington
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-08-01 至 2024-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2127235&HistoricalAwards=false
• 关键词: Collaborative; Research; OP; Meta; optical

In modern daily life, cameras are indispensable, and they truly serve an excellent purpose to capture a scene as perceived by a human eye. Digital photography became a disruptive technology when it was first introduced almost 30 years ago. From that time, cameras have undergone dramatic miniaturization. With these cameras readily available to consumers, professionals and hobbyists are able to experience how easily a photo can be captured, viewed, and shared. But many emerging applications in machine vision, robotics or internet of things require ever more advanced (smaller, lower power and intelligent) cameras. These cameras are expected not just to capture images, but also to provide information on how a machine must function, like for example in autonomous navigation. For this type of scene-understanding or object-detection problems, current systems employ bulky cameras combined with a computer or graphical processing unit. Unfortunately, most of these systems consume significant amounts of energy, and often are not optimized for specific tasks. By co-designing the hardware and software together, this project aims to create computational machine vision sensors, capable of low-power, low-latency operation and compact in size. The resulting sensors can revolutionize the field of autonomous navigation and machine vision. Furthermore, this project will improve the training and education of undergraduate and high school students, with a strong emphasis on including women and minority communities, in multi-disciplinary research in optics and machine learning. Through the PI’s active involvement with industrial laboratories working on automotive, imaging and augmented reality visors, the scientific results will be disseminated to a wider scientific audience via seminars, workshops, peer-reviewed publications, and conferences. There is a tremendous need for compact, low-power, and ubiquitous image sensors for applications in autonomous transportation, smart homes and cities, and the Internet of Things. Many of these machine vision applications require an electronic back-end to interpret the captured images or need more information than just the two-dimensional intensity information usually captured in cameras. Current approaches for solving these problems employ high-end, bulky cameras to capture high-quality images and then exploit computationally expensive and power-hungry computer vision algorithms. Both the size and power consumption of these imaging systems can be drastically reduced via co-optimizing the optics and computational imaging algorithms for specific applications, including depth sensing and directly solving higher-level computer vision tasks such as object segmentation, detection, and classification. This project aims to research and develop such a co-optimization algorithm for an optical front-end and complementary computational back end. The optical elements are implemented via high-efficiency dielectric meta-optics, where each scatterer constitutes a design parameter. Combining numerical simulation, device fabrication, and optical characterization, this project aims to develop an inverse design framework for optimizing the sensor’s meta-optics; expand the design framework to co-optimize both the meta-optics and computational algorithms without placing prohibitive constraints on intermediate representations, as well as fabricate and characterize the meta-optical sensors for 3D imaging and object detection.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

在现代日常生活中，相机是不可或缺的，它们确实具有捕捉人眼所感知的场景的绝佳用途，自从近 30 年前首次推出以来，相机已经成为一项颠覆性技术。随着这些相机可供消费者使用，专业人士和业余爱好者能够体验如何轻松地拍摄、查看和共享照片，但机器视觉、机器人或物联网领域的许多新兴应用需要更先进的技术（更小）。 、低功耗、智能化）这些相机不仅可以捕获图像，还可以提供有关机器如何运行的信息，例如对于这种类型的场景理解或物体检测问题，当前的系统采用了笨重的相机组合。不幸的是，这些系统中的大多数都会消耗大量的能量，并且通常没有针对特定任务进行优化，该项目旨在创建具有计算能力的计算机机器视觉传感器。低功耗、低延迟操作和紧凑型由此产生的传感器可以彻底改变自主导航和机器视觉领域，该项目将改善本科生和高中生的培训和教育，重点关注女性和少数民族社区参与多学科研究。通过 PI 积极参与汽车、成像和增强现实遮阳板的工业实验室，科学成果将通过研讨会、讲习班、同行评审出版物和会议传播给更广泛的科学受众。巨大的需要用于自动交通、智能家居和城市以及物联网应用的紧凑、低功耗和无处不在的图像传感器，其中许多机器视觉应用需要电子后端来解释捕获的图像或需要更多信息。目前解决这些问题的方法是使用高端、笨重的相机来捕获高质量图像，然后利用计算成本高且耗电的计算机视觉算法。这些成像系统的数量可以通过针对特定应用（包括深度传感和直接解决对象分割、检测和分类等更高级别的计算机视觉任务）共同优化光学和计算成像算法。该项目旨在研究和开发此类共同优化算法。光学前端和互补计算后端是通过高效介电元光学实现的，其中每个散射体构成一个设计参数，该项目旨在开发一种逆散射。设计框架优化传感器的元光学；扩展设计框架以共同优化元光学和计算算法，而不对中间表示施加禁止性限制，以及制造和表征用于 3D 成像和物体检测的元光学传感器。授予 NSF 的法定使命，并通过评估反映使用基金会的智力优点和更广泛的影响审查标准，被认为值得支持。

项目成果

Inverse designed extended depth of focus meta-optics for broadband imaging in the visible

用于可见光宽带成像的逆向设计扩展焦深元光学

• DOI: 10.1515/nanoph-2021-0431
• 发表时间: 2022-02
• 期刊: Nanophotonics
• 影响因子: 7.5
• 作者: Bayati, Elyas;Pestourie, Raphaël;Colburn, Shane;Lin, Zin;Johnson, Steven G.;Majumdar, Arka
• 通讯作者: Majumdar, Arka

Fast extended depth of focus meta-optics for varifocal functionality

用于变焦功能的快速扩展景深元光学器件

• DOI: 10.1364/prj.434681
• 发表时间: 2022-03
• 期刊: Photonics Research
• 影响因子: 7.6
• 作者: Whitehead, James E. M.;Zhan, Alan;Colburn, Shane;Huang, Luocheng;Majumdar, Arka
• 通讯作者: Majumdar, Arka

Large field-of-view short-wave infrared metalens for scanning fiber endoscopy

用于扫描纤维内窥镜的大视场短波红外超透镜

• DOI: 10.1117/1.jbo.28.9.094802
• 发表时间: 2023-03-09
• 期刊: Journal of Biomedical Optics
• 影响因子: 3.5
• 作者: N. Xie;Matthew D. Carson;Johannes E. Fröch;A. Majumdar;E. Seibel;K. Böhringer
• 通讯作者: K. Böhringer

Real time full-color imaging in a Meta-optical fiber endoscope

元光纤内窥镜中的实时全彩成像

• DOI: 10.1186/s43593-023-00044-4
• 发表时间: 2023-06
• 期刊: eLight
• 作者: Fröch, Johannes E.;Huang, Luocheng;Tanguy, Quentin A. A.;Colburn, Shane;Zhan, Alan;Ravagli, Andrea;Seibel, Eric J.;Böhringer, Karl F.;Majumdar, Arka
• 通讯作者: Majumdar, Arka

Large field-of-view thermal imaging via all-silicon meta-optics

通过全硅元光学器件实现大视场热成像

• DOI: 10.1364/ao.493555
• 发表时间: 2023-01
• 期刊: Applied Optics
• 影响因子: 1.9
• 作者: Wirth;Fröch, Johannes E.;Han, Zheyi;Huang, Luocheng;Mukherjee, Saswata;Zhou, Zhihao;Coppens, Zachary;Böhringer, Karl F.;Majumdar, Arka
• 通讯作者: Majumdar, Arka