CRII: CNS: System for Deploying Ultra Low-Latency Machine Learning Applications on Programmable Networks

CRII：CNS：在可编程网络上部署超低延迟机器学习应用程序的系统

• 批准号: 2245352
• 负责人: Sean Choi
• 金额: $ 17.42万
• 依托单位: Santa Clara University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-10-01 至 2025-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2245352&HistoricalAwards=false
• 关键词: CRII; CNS; System; Deploying; Ultra

Many modern applications, such as self-driving, security-threat detection, and image recognition rely on machine learning (ML) models, which are statistical models built using large amounts of data to automatically achieve solutions for a set of complex problems. ML models are often very complex, thus they often require very powerful computing hardware and large amounts of time, often taking minutes, to arrive at a solution. However, modern applications including those mentioned above require decisions to be made in milliseconds to be able to react to the changes in the environment. Therefore, a major challenge in machine learning is to develop methods and computer systems that allow ML models to be able to provide solutions to complex problems very quickly, while minimizing the amount of hardware that the models need. Solving such a challenge will not only increase the feasibility of using complex ML models for modern applications, but also, for example, provide potential improvements for defense systems that improve our national security. Furthermore, this research directly feeds into the development of new computer systems courses and provides opportunities for a number of undergraduates—many for the first time—to participate in research.This project focuses on solving the challenges in providing significant reduction in time—defined as latency—to provide solutions for applications that use ML models. An approach that will be explored by this project is for models that traditionally are run on a central processing unit (CPU) or a graphics processing unit (GPU) to run on a domain-specific architecture (DSA) called a network processing unit (NPU). NPUs are computational hardware that exist in networking devices, such as network interface cards (NICs), and act as the gateway to data that enters and leaves a computer. The main motivation for using NPUs is to mitigate the overhead of passing data to CPUs or GPUs, thereby reducing the latency, CPU cycles and memory spent on processing the data, while providing performance guarantees that come with the reduced need for context switching. The set of challenges for this approach are: (1) determining the types of ML applications that are feasible to be offloaded onto NPUs with measurable improvements; (2) programming and deploying applications on NPUs in an efficient and scalable manner; and (3) guaranteeing predictable performance with existing traffic. This project will focus on developing methods and a system for deploying ML applications on to NPUs and quantifying their benefits, focusing on existing, simpler ML models, such as decision trees and logistic regression, to show feasibility of its approach and to obtain preliminary metrics on performance improvements. The entire work will be released as open-source, reusable libraries, and applications for use by other researchers.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

自动驾驶、安全威胁检测和图像识别等许多现代应用都依赖于机器学习 (ML) 模型，这些模型是使用大量数据构建的统计模型，用于自动实现一组复杂问题的解决方案。通常非常复杂，因此通常需要非常强大的计算和大量时间（通常需要几分钟）才能得出解决方案。但是，包括上述应用程序在内的现代应用程序需要在几毫秒内做出决策，以便硬件能够做出反应。因此，环境的变化很大。机器学习面临的挑战是开发方法和计算机系统，使机器学习模型能够非常快速地为复杂问题提供解决方案，同时最大限度地减少模型所需的硬件数量，解决这一挑战不仅会增加使用的可行性。例如，这项研究还为国防系统提供了潜在的改进，从而提高了我们的国家安全。此外，这项研究直接促进了新计算机系统课程的开发，并为许多本科生提供了机会。第一次——参与研究。本项目重点致力于解决显着缩短时间（定义为延迟）的挑战，为使用 ML 模型的应用程序提供解决方案。该项目将探索的方法是针对传统上在中央处理单元 (CPU) 或计算机上运行的模型。在称为网络处理单元 (NPU) 的特定域架构 (DSA) 上运行的图形处理单元 (GPU) 是存在于网络设备（例如网络接口卡 (NIC)）中的计算硬件，并充当网络处理单元。进入和离开数据的网关计算机使用 NPU 的主要动机是减少将数据传递到 CPU 或 GPU 的开销，从而减少处理数据所花费的延迟、CPU 周期和内存，同时提供性能保证，从而减少上下文切换的需求。这种方法面临的挑战包括：(1) 确定可卸载到 NPU 上并进行可衡量改进的 ML 应用程序类型；(2) 以高效且可扩展的方式在 NPU 上编程和部署应用程序；该项目将重点开发用于将 ML 应用程序部署到 NPU 上并量化其优势的方法和系统，重点关注现有的更简单的 ML 模型（例如决策树和逻辑回归），以展示其可行性。方法并获得性能改进的初步指标。整个工作将作为开源、可重用的库和应用程序发布，供其他研究人员使用。该奖项是 NSF 的法定使命，并通过使用基金会的知识进行评估，被认为值得支持。优点和更广泛的影响审查标准。