CRII: CNS: System for Deploying Ultra Low-Latency Machine Learning Applications on Programmable Networks

CRII：CNS：在可编程网络上部署超低延迟机器学习应用程序的系统

• 批准号: 2245352
• 负责人: Sean Choi
• 金额: $ 17.42万
• 依托单位: Santa Clara University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-10-01 至 2025-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2245352&HistoricalAwards=false
• 关键词: CRII; CNS; System; Deploying; Ultra

Many modern applications, such as self-driving, security-threat detection, and image recognition rely on machine learning (ML) models, which are statistical models built using large amounts of data to automatically achieve solutions for a set of complex problems. ML models are often very complex, thus they often require very powerful computing hardware and large amounts of time, often taking minutes, to arrive at a solution. However, modern applications including those mentioned above require decisions to be made in milliseconds to be able to react to the changes in the environment. Therefore, a major challenge in machine learning is to develop methods and computer systems that allow ML models to be able to provide solutions to complex problems very quickly, while minimizing the amount of hardware that the models need. Solving such a challenge will not only increase the feasibility of using complex ML models for modern applications, but also, for example, provide potential improvements for defense systems that improve our national security. Furthermore, this research directly feeds into the development of new computer systems courses and provides opportunities for a number of undergraduates—many for the first time—to participate in research.This project focuses on solving the challenges in providing significant reduction in time—defined as latency—to provide solutions for applications that use ML models. An approach that will be explored by this project is for models that traditionally are run on a central processing unit (CPU) or a graphics processing unit (GPU) to run on a domain-specific architecture (DSA) called a network processing unit (NPU). NPUs are computational hardware that exist in networking devices, such as network interface cards (NICs), and act as the gateway to data that enters and leaves a computer. The main motivation for using NPUs is to mitigate the overhead of passing data to CPUs or GPUs, thereby reducing the latency, CPU cycles and memory spent on processing the data, while providing performance guarantees that come with the reduced need for context switching. The set of challenges for this approach are: (1) determining the types of ML applications that are feasible to be offloaded onto NPUs with measurable improvements; (2) programming and deploying applications on NPUs in an efficient and scalable manner; and (3) guaranteeing predictable performance with existing traffic. This project will focus on developing methods and a system for deploying ML applications on to NPUs and quantifying their benefits, focusing on existing, simpler ML models, such as decision trees and logistic regression, to show feasibility of its approach and to obtain preliminary metrics on performance improvements. The entire work will be released as open-source, reusable libraries, and applications for use by other researchers.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

许多现代应用程序，例如自动驾驶，安全威胁检测和图像识别都取决于机器学习（ML）模型，这些模型是使用大量数据构建的统计模型，以自动为一组复杂问题实现解决方案。 ML模型通常非常复杂，因此它们通常需要非常强大的计算硬件和大量时间（通常花费几分钟）才能达到解决方案。但是，现代应用程序包括上面提到的所需的决定，以毫秒来对环境的变化做出反应。因此，机器学习的主要挑战是开发方法和计算机系统，以使ML模型能够非常快速地为复杂问题提供解决方案，同时最大程度地减少模型所需的硬件量。解决这样的挑战不仅会增加对现代应用使用复杂的ML模型的可行性，而且还可以为改善我们国家安全的国防系统提供潜在的改进。此外，这项研究直接介绍了新的计算机系统课程的开发，并为许多本科生（首次参加研究）提供了机会。该项目着重于解决时间的大幅度降低（以延迟为延迟）为使用ML模型提供的解决方案。该项目将探索的一种方法是针对传统上在中央处理单元（CPU）或图形处理单元（GPU）上运行的模型，以在特定于域特异性体系结构（DSA）上运行，称为网络处理单元（NPU）。 NPU是网络设备（例如网络接口卡（NIC））中存在的计算硬件，并充当进入和离开计算机的数据的网关。使用NPU的主要动机是减轻将数据传递给CPU或GPU的开销，从而减少用于处理数据上的延迟，CPU周期和内存，同时提供对上下文切换的减少需求所带来的性能保证。这种方法的一系列挑战是：（1）确定可将可行的ML应用程序的类型卸载到NPU上，并进行可测量的改进； （2）以有效且可扩展的方式对NPU进行编程和部署应用程序； （3）保证通过现有流量提供可预测的性能。该项目将着重于开发方法和一个系统，用于将ML应用程序部署到NPU并量化其收益，重点关注现有的，更简单的ML模型，例如决策树和逻辑回归，以显示其方法的可行性并获得有关绩效改进的初步指标。整个工作将作为开源，可重复使用的图书馆以及其他研究人员使用的申请发行。该奖项反映了NSF的法定任务，并使用基金会的知识分子优点和更广泛的审查标准，认为值得通过评估来获得支持。