CMOS+X: Retinomorphic Infrared Imager with Sparsity-adaptive Machine-Learning Accelerator

CMOS X：具有稀疏自适应机器学习加速器的视网膜成像红外成像仪

• 批准号: 2318990
• 负责人: Tse Nga Ng
• 金额: $ 50万
• 依托单位: University of California-San Diego
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-08-15 至 2026-07-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2318990&HistoricalAwards=false
• 关键词: CMOS+X; Retinomorphic; Infrared; Imager; Sparsity

Real-time signal analysis plays a critical role in applications such as autonomous navigation and robotic guidance. However, conventional imaging systems are hindered by delays and high energy costs associated with transferring large amounts of data to centralized processors. To overcome those shortcomings, this project will embed computational elementals directly in the sensor arrays to locally run machine-learning algorithms that would provide capabilities for object recognition and motion tracking in real time and at low power. Such local in-sensor computing approach would offer significant reductions in energy consumption and delays by avoiding the data transfer bottlenecks. This project will co-optimize the designs of organic infrared sensors and silicon circuits, taking inspiration from the biological retina, which is highly sensitive to dynamic changes and well-suited for motion analysis. The proposed prototype is expected to reduce energy consumption by up to 100 times compared to conventional architectures, thereby paving the way for low-power machine learning. The integration research will contribute to advancing semiconductor manufacturing technologies within the United States and support workforce training. The research team will also engage in outreach activities to promote awareness of various career paths in science and engineering and the rewards of engineering careers.The goal of this research project is to integrate retinomorphic infrared sensors and silicon circuits in order to create a prototype imaging system with machine-learning capabilities for motion analysis. The design strategy assigns complementary roles to the organic sensors and silicon circuits: the retinomorphic sensors will generate highly sparse, feature-extracted data in both temporal and spatial domains, while the circuitries will use the sparsity to boost the overall system performance and power savings. The first research objective focuses on enhancing the sensor’s signal gain and adjusting the time constant to deliver streamlined data into the silicon processor. The second objective aims to optimize sparsity-adaptive architectures that can handle a wide range of sparsity levels and implement the circuit designs using 65 nm technology. The third objective involves establishing the processing workflow to integrate organic sensor arrays onto silicon chips and evaluating the functionalities of the imager by measuring the success rate of object tracking and classification. The resulting prototype will offer valuable insights into design guidelines for effectively balancing energy use, noise and variation tolerance, and latency in smart infrared imaging systems. The proposed imager, equipped with sophisticated yet energy-efficient machine learning capabilities, will have broad applicability across various fields including navigation, biomedical imaging, security, and machine vision applications.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

实时信号分析在自主导航和机器人指导等应用中起关键作用。但是，传统的成像系统受到与将大量数据传输到集中处理器相关的延迟和高能源成本的阻碍。为了克服这些缺点，该项目将直接将计算元素嵌入传感器阵列中，以在本地运行机器学习算法，这些算法将为对象识别和运动跟踪实时和低功率提供功能。这种局部发射内计算方法将通过避免数据传输瓶颈可显着减少能源消耗和延迟。该项目将在有机红外传感器和硅电路的设计中进行优化，并从生物视网膜中获得灵感，这对动态变化非常敏感，并且非常适合运动分析。与常规体系结构相比，提出的原型预计将减少100倍的能源消耗，从而为低功耗机器学习铺平了道路。整合研究将有助于推进美国境内的半导体制造技术并支持劳动力培训。研究团队还将从事外展活动，以促进对科学和工程学的各种职业道路的认识以及工程职业的回报。该研究项目的目的是将视网膜形态的红外传感器和硅电路整合在一起，以便创建一个原型成像系统与机器学习能力，以进行运动分析。设计策略将完整的角色分配给有机传感器和硅电路：视网膜形态传感器将在临时和空间域中产生高度稀疏的，特征提取的数据，而电路将利用稀疏来提高整体系统性能和功率。第一个研究目标侧重于增强传感器的信号增益并调整时间常数，以将流线型数据传递到硅处理器中。第二个目标旨在优化可以处理各种稀疏水平并使用65 nm技术实施电路设计的稀疏自适应体系结构。第三个目标涉及建立处理工作流程，以将有机传感器阵列整合到硅芯片上，并通过测量对象跟踪和分类的成功率来评估成像仪的功能。由此产生的原型将为设计指南提供有效平衡能量使用，噪声和变化耐受性以及智能红外成像系统延迟的宝贵见解。该提议的成像仪配备了复杂而节能的机器学习能力，将在包括导航，生物医学成像，安全性和机器视觉应用在内的各个领域具有广泛的适用性。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的智力和更广泛影响的评估来通过评估来获得支持的。