CAREER: Heterogeneous Neuromorphic and Edge Computing Systems for Realtime Machine Learning Technologies

职业：用于实时机器学习技术的异构神经形态和边缘计算系统

• 批准号: 2340249
• 负责人: Ramtin Mohammadizand
• 金额: $ 59.32万
• 依托单位: University of South Carolina at Columbia
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2024
• 资助国家: 美国
• 起止时间: 2024-05-01 至 2029-04-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2340249&HistoricalAwards=false
• 关键词: CAREER; Heterogeneous; Neuromorphic; Edge; Computing

Machine learning systems in robotics, self-driving cars, assistive technologies, and Internet-of-Things (IoT) applications require low-energy, real-time computation. Lower energy use ensures extended battery life for these battery-powered devices. This project focuses on American sign language translation to showcase its societal impact. To create practical sign language translation technology, multiple computer vision and language models are essential for seamless communication between sign language users and others. The aim is to deploy this on portable, wearable devices for on-demand use - a complex challenge. The research team will investigate breaking down these complex systems, distributing computation across interconnected tiny devices specialized in specific tasks. The outcome of this work can empower those with hearing and speech impairments, fostering inclusive communication and workforce diversity. Beyond sign language translation, the methodology and framework developed in this project can pave the way for real-time technology in social robotics and smart manufacturing, among other domains. This project involves various educational and outreach initiatives, including developing cross-disciplinary curricula, generating online educational resources, engaging both undergraduate and high school students in research, and collaborating with industry partners to promote social robotics for K-5 learning.This project aims to harness the combined capabilities of neuromorphic and edge computing to forge a heterogeneous machine learning system. Its primary goal is to enable computer vision and language models on resource- and energy-constrained devices at an unprecedented scale. It focuses on several key aspects: (1) developing hybrid models that merge the energy efficiency, temporal sparsity, and spatiotemporal processing of spiking neural networks with the global processing of transformer models for complex large-scale computer vision tasks, (2) creating a methodology to deploy large language models on edge devices by employing system-level innovations such as computational graph modifications, custom kernels, and mathematical refactoring, (3) designing a flexible edge artificial intelligence (AI) accelerator to overcome hardware limitations hindering real-time implementation of large transformer models at the edge, (4) seamlessly integrating a heterogeneous system of mobile processors, edge AI accelerators, and neuromorphic hardware for a comprehensive end-to-end solution. Throughout the project, rigorous investigation delves into critical trade-offs between bandwidth, accuracy, performance, and energy consumption.This project is jointly funded by the Software and Hardware Foundation (SHF) core research program and the Established Program to Stimulate Competitive Research (EPSCoR).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

机器人、自动驾驶汽车、辅助技术和物联网 (IoT) 应用中的机器学习系统需要低能耗的实时计算。较低的能耗可确保延长这些电池供电设备的电池寿命。该项目重点关注美国手语翻译，以展示其社会影响。为了创建实用的手语翻译技术，多种计算机视觉和语言模型对于手语用户和其他人之间的无缝沟通至关重要。目的是将其部署在便携式可穿戴设备上以供按需使用——这是一个复杂的挑战。研究团队将研究分解这些复杂的系统，将计算分布在专门执行特定任务的互连微型设备上。这项工作的成果可以增强听力和语言障碍人士的能力，促进包容性沟通和劳动力多样性。除了手语翻译之外，该项目开发的方法和框架还可以为社交机器人和智能制造等领域的实时技术铺平道路。该项目涉及各种教育和推广举措，包括开发跨学科课程、生成在线教育资源、让本科生和高中生参与研究，以及与行业合作伙伴合作推广用于 K-5 学习的社交机器人。该项目旨在利用神经形态和边缘计算的组合能力来打造异构机器学习系统。其主要目标是在资源和能源受限的设备上以前所未有的规模实现计算机视觉和语言模型。它重点关注几个关键方面：(1) 开发混合模型，将尖峰神经网络的能源效率、时间稀疏性和时空处理与复杂的大规模计算机视觉任务的变压器模型的全局处理相结合，(2) 创建一个通过采用计算图修改、自定义内核和数学重构等系统级创新在边缘设备上部署大型语言模型的方法，(3) 设计灵活的边缘人工智能 (AI) 加速器以克服阻碍实时实现的硬件限制大型变压器的(4) 无缝集成移动处理器、边缘人工智能加速器和神经形态硬件的异构系统，形成全面的端到端解决方案。在整个项目中，严格的调查深入探讨了带宽、准确性、性能和能耗之间的关键权衡。该项目由软件和硬件基金会 (SHF) 核心研究计划和刺激竞争研究既定计划 (EPSCoR) 共同资助）。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。