CAREER: Heterogeneous Neuromorphic and Edge Computing Systems for Realtime Machine Learning Technologies

职业：用于实时机器学习技术的异构神经形态和边缘计算系统

• 批准号: 2340249
• 负责人: Ramtin Mohammadizand
• 金额: $ 59.32万
• 依托单位: University of South Carolina at Columbia
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2024
• 资助国家: 美国
• 起止时间: 2024-05-01 至 2029-04-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2340249&HistoricalAwards=false
• 关键词: CAREER; Heterogeneous; Neuromorphic; Edge; Computing; CAREER; Heterogeneous; Neuromorphic; Edge; Computing

Machine learning systems in robotics, self-driving cars, assistive technologies, and Internet-of-Things (IoT) applications require low-energy, real-time computation. Lower energy use ensures extended battery life for these battery-powered devices. This project focuses on American sign language translation to showcase its societal impact. To create practical sign language translation technology, multiple computer vision and language models are essential for seamless communication between sign language users and others. The aim is to deploy this on portable, wearable devices for on-demand use - a complex challenge. The research team will investigate breaking down these complex systems, distributing computation across interconnected tiny devices specialized in specific tasks. The outcome of this work can empower those with hearing and speech impairments, fostering inclusive communication and workforce diversity. Beyond sign language translation, the methodology and framework developed in this project can pave the way for real-time technology in social robotics and smart manufacturing, among other domains. This project involves various educational and outreach initiatives, including developing cross-disciplinary curricula, generating online educational resources, engaging both undergraduate and high school students in research, and collaborating with industry partners to promote social robotics for K-5 learning.This project aims to harness the combined capabilities of neuromorphic and edge computing to forge a heterogeneous machine learning system. Its primary goal is to enable computer vision and language models on resource- and energy-constrained devices at an unprecedented scale. It focuses on several key aspects: (1) developing hybrid models that merge the energy efficiency, temporal sparsity, and spatiotemporal processing of spiking neural networks with the global processing of transformer models for complex large-scale computer vision tasks, (2) creating a methodology to deploy large language models on edge devices by employing system-level innovations such as computational graph modifications, custom kernels, and mathematical refactoring, (3) designing a flexible edge artificial intelligence (AI) accelerator to overcome hardware limitations hindering real-time implementation of large transformer models at the edge, (4) seamlessly integrating a heterogeneous system of mobile processors, edge AI accelerators, and neuromorphic hardware for a comprehensive end-to-end solution. Throughout the project, rigorous investigation delves into critical trade-offs between bandwidth, accuracy, performance, and energy consumption.This project is jointly funded by the Software and Hardware Foundation (SHF) core research program and the Established Program to Stimulate Competitive Research (EPSCoR).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

机器人技术中的机器学习系统，自动驾驶汽车，辅助技术和图像互联网（IoT）应用程序需要低能，实时计算。较低的能源使用可确保这些电池动力设备的延长电池寿命。该项目着重于美国手语翻译，以展示其社会影响。为了创建实用的手语翻译技术，多个计算机视觉和语言模型对于手语用户和其他人之间的无缝沟通至关重要。目的是将其部署在便携式可穿戴设备上，以进行按需使用 - 一个复杂的挑战。研究团队将调查分解这些复杂系统，在专门从事特定任务的互连的小型设备上分发计算。这项工作的结果可以增强那些有听力和言语障碍的人的能力，从而促进包容性的沟通和劳动力多样性。除了手语翻译外，该项目中开发的方法和框架还可以为社交机器人技术和智能制造中的实时技术铺平道路。该项目涉及各种教育和外展计划，包括开发跨学科课程，创造在线教育资源，吸引本科和高中生从事研究，并与行业合作伙伴合作，以促进K-5学习的社交机器人技术。该项目的目标是利用神经元素的合并能力，并为heledece ogge for Heledoseens songeque songeque sange songeque sange songece songeque sange songece songeque sange songece songeque sange songeque sange songece open helederogenogemenoge sange systone。它的主要目标是以前所未有的规模启用有关资源和能源约束设备的计算机视觉和语言模型。它重点介绍了几个关键方面：（1）开发合并能源效率，暂时性稀疏性以及尖峰神经网络的时空处理，并通过全球处理变压器模型的全球处理，用于复杂的大规模计算机视觉任务任务，（2）创建一种方法，从（3）设计灵活的边缘人工智能（AI）加速器，以克服硬件限制，阻碍了边缘大型变压器模型的实时实现，（4）无缝地集成了移动处理器，边缘AI加速器和神经形态硬件的异质系统，以实现全面的End End End End End End End End End-End End-End-End-End。在整个项目中，严格的调查研究了带宽，准确性，绩效和能源消耗之间的关键权衡。该项目由软件和硬件基金会（SHF）核心研究计划和既定计划（EPSCOR）共同资助。该奖项（EPSCOR）奖励NSF的法定任务，并反映了通过评估的范围来审查构成群体的范围，并已被评估范围的商业群体和范围的范围。