Collaborative Research: FuSe: Metaoptics-Enhanced Vertical Integration for Versatile In-Sensor Machine Vision

合作研究：FuSe：Metaoptics 增强型垂直集成，实现多功能传感器内机器视觉

• 批准号: 2328856
• 负责人: Yuhao Zhu
• 金额: $ 49.95万
• 依托单位: University of Rochester
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-10-01 至 2026-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2328856&HistoricalAwards=false
• 关键词: Collaborative; Research; FuSe; Metaoptics; Enhanced

Vision is perhaps the most important human perception, as the majority of the brain’s cognitive function is dedicated to processing visual information. Despite recent advancements, today’s vision sensors remain quite primitive when compared to the superior ability of human visual perception. Moreover, the rapid development of deep learning and artificial intelligence (AI) has unleashed a new wave of machine vision, where increasing amounts of image data are generated and consumed, not by humans, but by edge devices to perform intelligent tasks such as classification, recognition, and perception. Inspired by the biological system and motivated by the huge demand of machine vision, this project investigates an integrated and holistic approach to building versatile vision systems that can be tailored for domain-specific tasks. It aims to create a vertically-integrated design stack for vision sensors across optics, image sensors, and vision processors. The project is expected to herald a new paradigm of AI-driven vision systems and demonstrate technology to address pivotal engineering challenges from real-time visual adaptivity in self-driving cars to near-zero energy efficiency in persistent environmental monitoring. In addition, the project’s education and workforce development activities foster an open-source hardware community to boost accessibility and deepen collaboration beyond the traditional discipline divides, as well as to build up the capacity of domestic talents in vision sensor industry, critical to national security and supply chain safety. The research objective of this project is to create the scientific and engineering foundations for a novel machine vision system that explores the hybrid integration of nanophotonic metamaterials and complementary metal-oxide semiconductor (CMOS) circuits and synergistically leverages the intrinsic computing capability of computational metasurface and analog-domain encoder embedded inside the image sensors. Our principled approach to abstracting design knobs and modeling interactions and tradeoffs across the system layers and physical domains will inform future “More than Moore” multi-physics semiconductor device integration. We will delve into the key concept of optimally distributing computation along the processing pipeline with complementary intrinsic physical-domain operations. Our end-to-end design framework is deliberately created to bridge the divide between modeling/simulation infrastructure and design toolchains across multiple heterogeneous physical domains. The core principles of embedding machine-learning-enabled feature selection with optical/electrical vertical integration could have a major impact on the design of sensor-rich intelligent physical platforms where resource constraints coincide with strict latency requirements. The technology developed in this project will turbocharge AI-enabled hardware to satisfy the tremendous computational demand imposed by data proliferation, broadly benefiting a range of burgeoning industries such as machine vision as a service, smart IoT infrastructure, data-driven sensing and imaging.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

视觉可能是最重要的人类感知，因为大脑的大部分认知功能致力于处理视觉信息，尽管最近取得了进步，但与人类视觉感知的卓越能力相比，当今的视觉传感器仍然相当原始。深度学习和人工智能 (AI) 的结合掀起了机器视觉的新浪潮，越来越多的图像数据不是由人类而是由边缘设备生成和消耗，以执行分类、识别和感知等智能任务。受到生物系统的启发和动力为了满足机器视觉的巨大需求，该项目研究了一种集成的整体方法来构建可针对特定领域任务进行定制的多功能视觉系统，旨在为跨光学、图像传感器和视觉的视觉传感器创建垂直集成的设计堆栈。该项目预计将预示着人工智能驱动视觉系统的新范例，并展示解决从自动驾驶汽车的实时视觉自适应性到持续环境监测中接近零能源效率的关键工程挑战的技术。该项目的教育和劳动力发展活动培育开源硬件社区，以提高可访问性并深化超越传统学科界限的合作，并增强视觉传感器行业国内人才的能力，这对国家安全和供应链安全至关重要。该项目旨在为新型机器视觉系统奠定科学和工程基础，该系统探索纳米光子超材料和互补金属氧化物半导体（CMOS）电路的混合集成，并协同利用计算超表面和嵌入其中的模拟域编码器的固有计算能力这我们抽象设计旋钮以及建模跨系统层和物理域的交互和权衡的原则性方法将为未来的“超越摩尔”多物理场半导体器件集成提供信息，我们将深入研究沿系统优化分布计算的关键概念。我们特意创建了具有互补的内在物理域操作的处理管道，以弥合跨多个异构物理域的建模/仿真基础设施和设计工具链之间的鸿沟。具有光学/电气垂直集成的机器学习功能选择可能会对传感器丰富的智能物理平台的设计产生重大影响，在该平台中，资源限制与严格的延迟要求相一致，该项目中开发的技术将增强人工智能硬件的性能。满足数据激增带来的巨大计算需求，广泛惠及一系列新兴行业，例如机器视觉即服务、智能物联网基础设施、数据驱动的传感和成像。该奖项反映了 NSF 的法定使命，并被认为值得通过以下方式获得支持：使用基金会的评估智力价值和更广泛的影响审查标准。