CAREER:Bionic Eye: Heterogeneous Integration of Hemispherical Image Sensor with Artificial Neural Network

职业：仿生眼：半球图像传感器与人工神经网络的异构集成

• 批准号: 1942868
• 负责人: Kyusang Lee
• 金额: $ 50万
• 依托单位: University of Virginia Main Campus
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-02-01 至 2025-01-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1942868&HistoricalAwards=false
• 关键词: CAREER; Bionic; Eye; Heterogeneous; Integration

Non-Technical:Vision is one of the most crucial senses for humans when evaluating a scene. Natural imaging systems such as the eye provide an aberration-free image with a wide field of view. Artificial imaging systems can be made lighter, smaller, and with minimal optical aberrations by curving the focal plane array to match the curvature of the lens. Such a conformal architecture is analogous to the shape of the human eye. Furthermore, integration of an artificial synapse in image sensors will enable to processing of the high volumes of data acquired from a scene at low power. This approach is inspired by the interaction between the eye and visual cortex. In this project, the PI will design integrated hemispherical image sensors with an embedded neuromorphic chip that allows edge computing at hardware level with low-power operation. The system will provide wide-angle, aberration free images for various applications such as vehicle navigation, threat detection, and object identification. This will greatly facilitate further development of broad smart sensor applications combined with simplified AI technology. The PI will create introductory lab modules that connect basic material science and device physics to system level integration, allowing students to connect basic science with real-life applications.Technical:In this program, the PI will demonstrate an artificial retina integrated with neuromorphic circuits that consumes extremely low power. The PI will focus on designing and fabricating a thin-film InGaAs based hemispherical focal plane array and memrisror based artificial synapses that provides a wide field of view, simultaneous localization and mapping (SLAM), data reduction, ranging capability adaptable to variable and near infrared light levels while providing object recognition capabilities. This will be enabled by a combination of various unique technologies; from materials growth and lift-off, flexible device fabrication to heterogeneous and system level integration. The fabricated thin-film compound semiconductor based hemispherical image sensor array using remote epitaxy technique will be integrated with novel computing architecture based on emerging non-volatile resistive switching devices, providing an ideal implementation of a synaptic weight in artificial neural networks. This integrated image sensor will enable edge extraction by applying hexagonal kernel to the honeycomb image sensor array, and object recognition via a single layer of fully connected convolution neural network (CNN) similar to the human retina and visual cortex. Eventually, this proposed project will provide imagers that would enhance situational awareness and recognition with minimal power consumption in a power-constrained environment that is of central importance to robotics, autonomous driving, and military applications etc.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

非技术：视觉是评估场景时人类最关键的感觉之一。诸如眼睛之类的天然成像系统提供了无像差图像，并具有广阔的视野。通过弯曲焦平面阵列以匹配镜头的曲率，可以使人工成像系统变得更轻，较小，并且具有最小的光学畸变。这种保形结构类似于人眼的形状。此外，图像传感器中人工突触的集成将使从低功率以低功率的场景获取的大量数据处理。这种方法的灵感来自眼睛和视觉皮层之间的相互作用。在这个项目中，PI将使用嵌入式神经形态芯片设计集成的半球形图像传感器，该芯片允许使用低功率操作在硬件级别的边缘计算。该系统将为各种应用（例如车辆导航，威胁检测和对象识别）提供广角，无像差图像。这将极大地促进广泛的智能传感器应用程序与简化的AI技术相结合的进一步开发。 PI将创建介绍性实验室模块，将基本材料科学和设备物理与系统级集成联系起来，使学生能够将基础科学与现实生活应用联系起来。技术：在该程序中，PI将展示与神经形态电路集成的人工视网膜，消耗极低的功率。 PI将着重于设计和制造基于薄膜INGAAS的半球局灶性平面阵列和基于Memrisror的人造突触，这些突触提供了广泛的视野，同时定位和映射（SLAM），数据降低，可提供可变功能的功能，并提供近距离的光线，同时提供对象识别能力。这将通过各种独特技术的结合来实现。从材料的生长和提升，灵活的设备制造到异质和系统水平集成。制造的薄膜化合物半导体基于远程外部附加技术的基于半球形图像传感器阵列将与基于新兴的非挥发性电阻开关设备的新型计算体系结构集成，从而提供了人工神经网络中突触重量的理想实现。该集成的图像传感器将通过将六边形内核应用于蜂窝图像传感器阵列，并通过类似于人类视网膜和视觉皮层的单层将六角内核和对象识别来实现边缘提取。最终，这个提议的项目将提供图像器，以在功率约束的环境中以最少的功耗来增强情境意识和认可，这对机器人技术，自动驾驶和军事应用至关重要，这是NSF的法定任务，这反映了NSF的法定任务，并通过该基金会的知识优点和广泛的效果来评估，这是值得通过评估来获得的。

项目成果

Non-Line-of-Sight Detection Based on Neuromorphic Time-of-Flight Sensing

• DOI: 10.1021/acsphotonics.3c00448
• 发表时间: 2023-06
• 期刊: ACS Photonics
• 影响因子: 7
• 作者: Minseong Park;Yuan Yuan-Yuan;Y. Baek;B. Bae;Bo-In Park;Young Hoon Kim;Nicholas Lin;J. Heo;Kyusang Lee

2D materials-assisted heterogeneous integration of semiconductor membranes toward functional devices

• DOI: 10.1063/5.0122768
• 发表时间: 2022-11
• 期刊: Journal of Applied Physics
• 影响因子: 3.2
• 作者: Minseong Park;B. Bae;Taegeon Kim;H. Kum;Kyusang Lee

Quantized Neural Network via Synaptic Segregation Based on Ternary Charge‐Trap Transistors

• DOI: 10.1002/aelm.202300303
• 发表时间: 2023-09
• 期刊: Advanced Electronic Materials
• 影响因子: 6.2
• 作者: Y. Baek;B. Bae;Jeongyong Yang;Doeon Lee;H. Lee;Minseong Park;Taegeon Kim;Sihwan Kim;Bo-In Park;Geonwook Yoo;Kyusang Lee

Efficient Defect Identification via Oxide Memristive Crossbar Array Based Morphological Image Processing

• DOI: 10.1002/aisy.202000202
• 发表时间: 2021-02-01
• 期刊: ADVANCED INTELLIGENT SYSTEMS
• 影响因子: 7.4
• 作者: Lee, Hee Sung;Baek, Yongmin;Lee, Kyusang
• 通讯作者: Lee, Kyusang

Fundamentals and applications of mixed-dimensional heterostructures

• DOI: 10.1063/5.0097804
• 发表时间: 2022-06-01
• 期刊: APL MATERIALS
• 影响因子: 6.1
• 作者: Lee, Kyusang;Duan, Xiangfeng;Kim, Jeehwan
• 通讯作者: Kim, Jeehwan