Collaborative Research: OP: Meta-optical Computational Image Sensors

合作研究：OP：元光学计算图像传感器

• 批准号: 2127331
• 负责人: Felix Heide
• 金额: $ 27.5万
• 依托单位: Princeton University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-08-01 至 2024-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2127331&HistoricalAwards=false
• 关键词: Collaborative; Research; OP; Meta; optical

In modern daily life, cameras are indispensable, and they truly serve an excellent purpose to capture a scene as perceived by a human eye. Digital photography became a disruptive technology when it was first introduced almost 30 years ago. From that time, cameras have undergone dramatic miniaturization. With these cameras readily available to consumers, professionals and hobbyists are able to experience how easily a photo can be captured, viewed, and shared. But many emerging applications in machine vision, robotics or internet of things require ever more advanced (smaller, lower power and intelligent) cameras. These cameras are expected not just to capture images, but also to provide information on how a machine must function, like for example in autonomous navigation. For this type of scene-understanding or object-detection problems, current systems employ bulky cameras combined with a computer or graphical processing unit. Unfortunately, most of these systems consume significant amounts of energy, and often are not optimized for specific tasks. By co-designing the hardware and software together, this project aims to create computational machine vision sensors, capable of low-power, low-latency operation and compact in size. The resulting sensors can revolutionize the field of autonomous navigation and machine vision. Furthermore, this project will improve the training and education of undergraduate and high school students, with a strong emphasis on including women and minority communities, in multi-disciplinary research in optics and machine learning. Through the PI’s active involvement with industrial laboratories working on automotive, imaging and augmented reality visors, the scientific results will be disseminated to a wider scientific audience via seminars, workshops, peer-reviewed publications, and conferences. There is a tremendous need for compact, low-power, and ubiquitous image sensors for applications in autonomous transportation, smart homes and cities, and the Internet of Things. Many of these machine vision applications require an electronic back-end to interpret the captured images or need more information than just the two-dimensional intensity information usually captured in cameras. Current approaches for solving these problems employ high-end, bulky cameras to capture high-quality images and then exploit computationally expensive and power-hungry computer vision algorithms. Both the size and power consumption of these imaging systems can be drastically reduced via co-optimizing the optics and computational imaging algorithms for specific applications, including depth sensing and directly solving higher-level computer vision tasks such as object segmentation, detection, and classification. This project aims to research and develop such a co-optimization algorithm for an optical front-end and complementary computational back end. The optical elements are implemented via high-efficiency dielectric meta-optics, where each scatterer constitutes a design parameter. Combining numerical simulation, device fabrication, and optical characterization, this project aims to develop an inverse design framework for optimizing the sensor’s meta-optics; expand the design framework to co-optimize both the meta-optics and computational algorithms without placing prohibitive constraints on intermediate representations, as well as fabricate and characterize the meta-optical sensors for 3D imaging and object detection.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

在现代的日常生活中，相机是必不可少的，它们确实是捕捉人眼所感知的场景的绝佳目的。大约30年前，数码摄影首次推出时，它成为了一项破坏性技术。从那时起，摄像机已经进行了戏剧性的小型化。有了这些摄像机，消费者很容易获得，专业人士和业余爱好者就可以体验到如何轻松捕获，查看和共享照片。但是，机器视觉，机器人技术或物联网中的许多新兴应用都需要更高级（较小，较小的功率和智能）摄像机。期望这些摄像机不仅捕获图像，还可以提供有关机器必须运行的信息，例如在自动导航中。对于这种类型的场景理解或对象检测问题，当前系统员工笨重的相机与计算机或图形处理单元相结合。不幸的是，这些系统中的大多数都消耗了大量的能量，并且通常不会针对特定任务进行优化。通过将硬件和软件共同设计，该项目旨在创建能够低功率，低延迟操作和尺寸紧凑的计算机视觉传感器。最终的传感器可以彻底改变自主导航和机器视觉的领域。此外，该项目将改善本科生和高中生的培训和教育，重点包括妇女和少数民族社区，在光学和机器学习的多学科研究中。通过PI积极参与从事汽车，成像和增强现实遮阳板从事的工业实验室，科学的结果将通过半默默无闻，讲习班，经过同行评审的出版物和会议来传播给更广泛的科学受众。对于自动运输，智能家居和城市以及物联网应用程序以及物联网的应用，非常需要紧凑，低功率和无处不在的图像传感器。这些机器视觉应用中的许多都需要电子后端来解释捕获的图像或需要更多信息，而不仅仅是相机中通常捕获的二维强度信息。解决这些问题的当前方法采用了高端，庞大的相机来捕获高质量的图像，然后利用计算昂贵且渴望强力的计算机视觉算法。这些成像系统的大小和功耗都可以通过对特定应用的光学和计算成像算法进行优化，包括深度敏感性以及直接求解更高级别的计算机视觉任务，例如对象细分，检测和分类。该项目旨在研究和开发用于光学前端和互补计算后端的这种合作算法。光学元素是通过高效率词典元访问实现的，其中每个散点子构成设计参数。该项目结合了数值仿真，设备制造和光学表征，旨在开发一个逆设计框架，以优化传感器的元视图；扩展设计框架，以优化元元素和计算算法，而无需对中间表示的禁止约束，并为3D成像和对象检测的元光传感器进行构建和表征。本体奖项通过评估的构成效果，反映了NSF诚实的范围，该奖项反映了诚实的构成群体的范围。