Integrating Federated Split Neural Network with Artificial Stereoscopic Compound Eyes for Optical Flow Sensing in 3D Space with Precision

将联合分裂神经网络与人工立体复眼相结合，实现 3D 空间中的精确光流传感

• 批准号: 2332060
• 负责人: Kyusang Lee
• 金额: $ 40万
• 依托单位: University of Virginia Main Campus
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2024
• 资助国家: 美国
• 起止时间: 2024-10-01 至 2027-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2332060&HistoricalAwards=false
• 关键词: Integrating; Federated; Split; Neural; Network

This project is aiming to develop an innovative image sensor inspired by arthropod eyes, featuring a wide field of view, high-speed operation, and efficient object tracking. These smart sensors can dramatically expand the field of view, enhance response speeds, and operate with energy efficiency. Central to this advancement is the fusion of photodiodes with artificial synapses and mimicry of geometrical shape, a pairing that mirrors the biological processes of neural networks within the insect eye. This configuration not only facilitates the rapid processing of visual information but also reduces the energy required to do so, marking a significant step forward from traditional planar imaging systems. Unlike conventional systems, the sensor arrays will adopt hemispherical designs, inspired by nature's own solution to wide-angle and efficient vision. This geometric optimization is crucial for capturing the patterns of movement across a visual scene with enhanced accuracy and depth, making the technology invaluable for applications that require precise motion detection and spatial awareness. To process the high-dimensional data captured by these sensors, the project further introduces a specialized neural network architecture. This approach divides data processing tasks between the sensors and a central processing unit, ensuring swift, efficient, and precise analysis. Such a configuration is particularly suited for dynamic environments where rapid decision-making is essential. By emulating the intricate vision systems found in nature, it offers a glimpse into a future where technology and biology converge, providing impactful solutions. The potential applications of this work span across various sectors, promising to enhance the efficiency, and capabilities of systems in robotics, autonomous driving, and beyond, thereby advancing the progress of science and technology for the benefit of society.The goal of this project is to develop and integrate a novel artificial compound eye system capable of advanced in-situ object tracking and depth perception. This system leverages the unique advantages of a photodiode integrated with an artificial synaptic device, embodying a computational layer for immediate data processing akin to biological synaptic functions. The hardware design, inspired by the hemispherical structure of arthropod eyes, enables a wide field of view and rapid image acquisition. To address the challenges of processing high-dimensional visual data efficiently, a communication, storage, and energy-efficient federated split learning framework will be employed. This framework optimizes data processing by distributing computational tasks between the sensor and a centralized processing unit, significantly enhancing the system's real-time object tracking capabilities. By integrating advanced hardware with software algorithms, this project aims to create a system that not only advances the scientific understanding of bio-inspired imaging but also offers practical solutions for real-world applications. The successful execution of this project is expected to set a new standard in optical sensing technology, contributing significantly to the fields of computer vision and neuromorphic computing.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目旨在开发一种受节肢动物眼睛启发的创新图像传感器，具有宽视野、高速运行和高效的物体跟踪功能。这些智能传感器可以极大地扩展视野、提高响应速度并提高能源效率。这一进步的核心是光电二极管与人工突触和几何形状模仿的融合，这种配对反映了昆虫眼内神经网络的生物过程。这种配置不仅有利于视觉信息的快速处理，而且还减少了所需的能量，标志着传统平面成像系统向前迈出了一大步。与传统系统不同，传感器阵列将采用半球形设计，其灵感来自大自然自身的广角和高效视觉解决方案。这种几何优化对于以更高的精度和深度捕获视觉场景中的运动模式至关重要，使得该技术对于需要精确运动检测和空间感知的应用而言具有无价的价值。为了处理这些传感器捕获的高维数据，该项目进一步引入了专门的神经网络架构。这种方法将数据处理任务分配给传感器和中央处理单元，确保快速、高效和精确的分析。这种配置特别适合需要快速决策的动态环境。通过模拟自然界中复杂的视觉系统，它可以让我们一睹技术和生物学融合的未来，从而提供有影响力的解决方案。这项工作的潜在应用跨越各个领域，有望提高机器人、自动驾驶等系统的效率和能力，从而推动科学技术的进步，造福社会。该项目的目标是开发和集成一种新型人工复眼系统，能够进行先进的原位物体跟踪和深度感知。该系统利用了与人工突触设备集成的光电二极管的独特优势，体现了类似于生物突触功能的即时数据处理的计算层。硬件设计的灵感来自节肢动物眼睛的半球形结构，可实现宽视野和快速图像采集。为了解决高效处理高维视觉数据的挑战，将采用通信、存储和节能的联邦分裂学习框架。该框架通过在传感器和集中处理单元之间分配计算任务来优化数据处理，显着增强系统的实时对象跟踪能力。通过将先进的硬件与软件算法相集成，该项目旨在创建一个系统，不仅可以增进对仿生成像的科学理解，还可以为现实世界的应用提供实用的解决方案。该项目的成功执行预计将树立光学传感技术的新标准，为计算机视觉和神经形态计算领域做出重大贡献。该奖项反映了 NSF 的法定使命，并通过利用基金会的智力优势进行评估，认为值得支持以及更广泛的影响审查标准。