Collaborative Research: FuSe: Metaoptics-Enhanced Vertical Integration for Versatile In-Sensor Machine Vision

合作研究:FuSe:Metaoptics 增强型垂直集成,实现多功能传感器内机器视觉

基本信息

  • 批准号:
    2416375
  • 负责人:
  • 金额:
    $ 110万
  • 依托单位:
  • 依托单位国家:
    美国
  • 项目类别:
    Continuing Grant
  • 财政年份:
    2023
  • 资助国家:
    美国
  • 起止时间:
    2023-10-15 至 2026-09-30
  • 项目状态:
    未结题

项目摘要

Vision is perhaps the most important human perception, as the majority of the brain’s cognitive function is dedicated to processing visual information. Despite recent advancements, today’s vision sensors remain quite primitive when compared to the superior ability of human visual perception. Moreover, the rapid development of deep learning and artificial intelligence (AI) has unleashed a new wave of machine vision, where increasing amounts of image data are generated and consumed, not by humans, but by edge devices to perform intelligent tasks such as classification, recognition, and perception. Inspired by the biological system and motivated by the huge demand of machine vision, this project investigates an integrated and holistic approach to building versatile vision systems that can be tailored for domain-specific tasks. It aims to create a vertically-integrated design stack for vision sensors across optics, image sensors, and vision processors. The project is expected to herald a new paradigm of AI-driven vision systems and demonstrate technology to address pivotal engineering challenges from real-time visual adaptivity in self-driving cars to near-zero energy efficiency in persistent environmental monitoring. In addition, the project’s education and workforce development activities foster an open-source hardware community to boost accessibility and deepen collaboration beyond the traditional discipline divides, as well as to build up the capacity of domestic talents in vision sensor industry, critical to national security and supply chain safety. The research objective of this project is to create the scientific and engineering foundations for a novel machine vision system that explores the hybrid integration of nanophotonic metamaterials and complementary metal-oxide semiconductor (CMOS) circuits and synergistically leverages the intrinsic computing capability of computational metasurface and analog-domain encoder embedded inside the image sensors. Our principled approach to abstracting design knobs and modeling interactions and tradeoffs across the system layers and physical domains will inform future “More than Moore” multi-physics semiconductor device integration. We will delve into the key concept of optimally distributing computation along the processing pipeline with complementary intrinsic physical-domain operations. Our end-to-end design framework is deliberately created to bridge the divide between modeling/simulation infrastructure and design toolchains across multiple heterogeneous physical domains. The core principles of embedding machine-learning-enabled feature selection with optical/electrical vertical integration could have a major impact on the design of sensor-rich intelligent physical platforms where resource constraints coincide with strict latency requirements. The technology developed in this project will turbocharge AI-enabled hardware to satisfy the tremendous computational demand imposed by data proliferation, broadly benefiting a range of burgeoning industries such as machine vision as a service, smart IoT infrastructure, data-driven sensing and imaging.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
视觉可能是人类最重要的看法,因为大脑的认知功能大部分都致力于处理视觉信息。尽管最近取得了进步,但与人类视觉感知的卓越能力相比,当今的视力传感器仍然非常原始。此外,深度学习和人工智能(AI)的快速发展已被释放出新的机器视觉浪潮,其中越来越多的图像数据是由人类生成和消耗的,而是通过边缘设备来执行智能任务,例如分类,识别和感知。受生物系统的启发,并受到机器视觉需求的巨大需求的启发,该项目研究了一种综合而整体的方法来建立多功能视觉系统,该方法可以针对特定于领域的任务进行量身定制。它旨在为跨光学,图像传感器和视觉处理器垂直整合的设计堆栈创建一个垂直整合的设计堆栈。预计该项目将预示着AI驱动的视觉系统的新范式,并展示了从自动驾驶汽车的实时视觉适应性到接近零的能源效率的技术挑战,以应对持续的环境监测。此外,该项目的教育和劳动力发展活动促进了一个开源硬件社区,以提高可访问性并加深协作超出传统学科的鸿沟,并建立国内人才在视觉传感器行业中对国家安全和供应链安全至关重要的能力。该项目的研究目标是为新型机器视觉系统创建科学和工程基础,该系统探讨了纳米光材料超材料和互补的金属氧化物半导体(CMOS)电路的混合整合,并协同利用了计算上的上层和模拟域的固有计算能力。我们在系统层和物理域上抽象设计旋钮以及建模相互作用和权衡的主要方法将为未来提供“超过摩尔”多物理学半导体设备的集成。我们将深入研究沿处理管道最佳分配计算的关键概念,并使用互补的内在物理域操作。我们故意创建了我们的端到端设计框架,以弥合建模/仿真基础架构与设计工具之间在多个异构物理领域之间进行的鸿沟。具有光学/电气垂直整合的嵌入机器学习功能选择的核心原理可能会对富含传感器的智能物理平台的设计产生重大影响,在这些设计中,资源约束与严格的延迟要求一致。该项目中开发的技术将启用涡轮增压硬件,以满足数据增殖所施加的巨大计算需求,从而使一系列新兴行业(如机器视觉,智能的IoT基础架构,数据驱动的敏感性和成像)受益于众多奖励,这反映了NSF的众多启发,这表明了NSF的法定任务和成像,从而受益于机器视觉,智能物联网基础架构和成像。 标准。

项目成果

期刊论文数量(1)
专著数量(0)
科研奖励数量(0)
会议论文数量(0)
专利数量(0)
Invited Paper: Learned In-Sensor Visual Computing: From Compression to Eventification
特邀论文:学习传感器内视觉计算:从压缩到事件化
{{ item.title }}
{{ item.translation_title }}
  • DOI:
    {{ item.doi }}
  • 发表时间:
    {{ item.publish_year }}
  • 期刊:
  • 影响因子:
    {{ item.factor }}
  • 作者:
    {{ item.authors }}
  • 通讯作者:
    {{ item.author }}

数据更新时间:{{ journalArticles.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ monograph.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ sciAawards.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ conferencePapers.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ patent.updateTime }}

Xuan Zhang其他文献

Impact of meteorological factors on the incidence of influenza in Beijing: A 35-year retrospective study based on Yunqi theory
气象因素对北京市流感发病的影响——基于云气理论的35年回顾性研究
Does collaborative learning design align with enactment? An innovative method of evaluating the alignment in the CSCL context
协作学习设计是否与制定相一致?
Attribute Sampling Plan for Submitted Lots Based on Prior Information and Bayesian Approach
基于先验信息和贝叶斯方法的提交批次的属性抽样计划
  • DOI:
    10.3390/math12111692
  • 发表时间:
    2024
  • 期刊:
  • 影响因子:
    2.4
  • 作者:
    Jing Zhao;Fengyun Zhang;Xuan Zhang;Yuping Hu;Wenxing Ding
  • 通讯作者:
    Wenxing Ding
Influence of Wind-Generated Wave Action on Mountain Reservoir Bank Collapse: A Case Study at the Lancang River, Western China
风力波浪作用对山地水库溃决的影响——以澜沧江为例
  • DOI:
    10.2113/2022/6427717
  • 发表时间:
    2022-01
  • 期刊:
  • 影响因子:
    2.4
  • 作者:
    Xuan Zhang;Qingxiang Meng;Faming Zhang
  • 通讯作者:
    Faming Zhang
Intracation and Interanion−Cation Charge-Transfer Properties of Tetrathiafulvalene-Bismuth-Halide Hybrids
四硫富瓦烯-铋-卤化物杂化物的离子间和阴离子间阳离子电荷转移特性
  • DOI:
    10.1021/acs.inorgchem.8b01692
  • 发表时间:
    2018
  • 期刊:
  • 影响因子:
    4.6
  • 作者:
    Xuan Zhang;Ping Zhang;Yi-Gang Weng;Zheng-Zhen Tang;Qin-Yu Zhu;Jie Dai
  • 通讯作者:
    Jie Dai

Xuan Zhang的其他文献

{{ item.title }}
{{ item.translation_title }}
  • DOI:
    {{ item.doi }}
  • 发表时间:
    {{ item.publish_year }}
  • 期刊:
  • 影响因子:
    {{ item.factor }}
  • 作者:
    {{ item.authors }}
  • 通讯作者:
    {{ item.author }}

{{ truncateString('Xuan Zhang', 18)}}的其他基金

Collaborative Research: FuSe: Metaoptics-Enhanced Vertical Integration for Versatile In-Sensor Machine Vision
合作研究:FuSe:Metaoptics 增强型垂直集成,实现多功能传感器内机器视觉
  • 批准号:
    2328855
  • 财政年份:
    2023
  • 资助金额:
    $ 110万
  • 项目类别:
    Continuing Grant
Atmospheric Lifecycle of Highly Oxygenated Multifunctional Compounds
高含氧多功能化合物的大气生命周期
  • 批准号:
    2131199
  • 财政年份:
    2021
  • 资助金额:
    $ 110万
  • 项目类别:
    Standard Grant
CAREER: Neural Network-Inspired Information Processing Beyond the Binary Digital Abstraction
职业:超越二进制数字抽象的神经网络启发信息处理
  • 批准号:
    1942900
  • 财政年份:
    2020
  • 资助金额:
    $ 110万
  • 项目类别:
    Continuing Grant
CPS: Medium: Modular Power Orchestration at the Meso-scale
CPS:中:中观规模的模块化电源编排
  • 批准号:
    1739643
  • 财政年份:
    2017
  • 资助金额:
    $ 110万
  • 项目类别:
    Standard Grant
CRII: SaTC: Investigation of Side-Channel Attack Vulnerability in Near-Threshold Computing Systems
CRII:SaTC:近阈值计算系统中的侧通道攻击漏洞调查
  • 批准号:
    1657562
  • 财政年份:
    2017
  • 资助金额:
    $ 110万
  • 项目类别:
    Standard Grant

相似国自然基金

支持二维毫米波波束扫描的微波/毫米波高集成度天线研究
  • 批准号:
    62371263
  • 批准年份:
    2023
  • 资助金额:
    52 万元
  • 项目类别:
    面上项目
腙的Heck/脱氮气重排串联反应研究
  • 批准号:
    22301211
  • 批准年份:
    2023
  • 资助金额:
    30 万元
  • 项目类别:
    青年科学基金项目
水系锌离子电池协同性能调控及枝晶抑制机理研究
  • 批准号:
    52364038
  • 批准年份:
    2023
  • 资助金额:
    33 万元
  • 项目类别:
    地区科学基金项目
基于人类血清素神经元报告系统研究TSPYL1突变对婴儿猝死综合征的致病作用及机制
  • 批准号:
    82371176
  • 批准年份:
    2023
  • 资助金额:
    49 万元
  • 项目类别:
    面上项目
FOXO3 m6A甲基化修饰诱导滋养细胞衰老效应在补肾法治疗自然流产中的机制研究
  • 批准号:
    82305286
  • 批准年份:
    2023
  • 资助金额:
    30 万元
  • 项目类别:
    青年科学基金项目

相似海外基金

Collaborative Research: FuSe: R3AP: Retunable, Reconfigurable, Racetrack-Memory Acceleration Platform
合作研究:FuSe:R3AP:可重调、可重新配置、赛道内存加速平台
  • 批准号:
    2328975
  • 财政年份:
    2024
  • 资助金额:
    $ 110万
  • 项目类别:
    Continuing Grant
Collaborative Research: FuSe: R3AP: Retunable, Reconfigurable, Racetrack-Memory Acceleration Platform
合作研究:FuSe:R3AP:可重调、可重新配置、赛道内存加速平台
  • 批准号:
    2328973
  • 财政年份:
    2024
  • 资助金额:
    $ 110万
  • 项目类别:
    Continuing Grant
Collaborative Research: FuSe: R3AP: Retunable, Reconfigurable, Racetrack-Memory Acceleration Platform
合作研究:FuSe:R3AP:可重调、可重新配置、赛道内存加速平台
  • 批准号:
    2328972
  • 财政年份:
    2024
  • 资助金额:
    $ 110万
  • 项目类别:
    Continuing Grant
Collaborative Research: FuSe: R3AP: Retunable, Reconfigurable, Racetrack-Memory Acceleration Platform
合作研究:FuSe:R3AP:可重调、可重新配置、赛道内存加速平台
  • 批准号:
    2328974
  • 财政年份:
    2024
  • 资助金额:
    $ 110万
  • 项目类别:
    Continuing Grant
Collaborative Research: FuSe: Indium selenides based back end of line neuromorphic accelerators
合作研究:FuSe:基于硒化铟的后端神经形态加速器
  • 批准号:
    2328741
  • 财政年份:
    2023
  • 资助金额:
    $ 110万
  • 项目类别:
    Continuing Grant
{{ showInfoDetail.title }}

作者:{{ showInfoDetail.author }}

知道了