CRII: OAC: High-Efficiency Serverless Computing Systems for Deep Learning: A Hybrid CPU/GPU Architecture

CRII：OAC：用于深度学习的高效无服务器计算系统：混合 CPU/GPU 架构

基本信息

批准号：
2153502
负责人：
Hao Wang
金额：
$ 17.49万
依托单位：
Louisiana State University
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2022
资助国家：
美国
起止时间：
2022-05-01 至 2025-04-30
项目状态：
未结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=2153502&HistoricalAwards=false
关键词：
CRII OAC Efficiency Serverless Computing

项目摘要

This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2).Next-generation serverless cloud computing provides developers with simplified access to server management and administration, including event-driven execution, fine-grained resource provisioning, auto-scaling, and pay-as-you-go billing. The machine learning community is taking advantage of these benefits of serverless cloud computing to ease the development and deployment of deep learning (DL) applications. However, existing serverless computing platforms lack efficient support for GPUs, impeding DL practitioners from utilizing serverless computing for large-scale applications. This project will develop an efficient serverless computing platform with a hybrid CPU/GPU architecture to accelerate DL application development and deployment. The goal is to advance cutting-edge methodologies in both deep learning and serverless computing, which will result in a significant leap forward to benefit DL practitioners, DL users, and providers of cloud computing infrastructures, contributing to science advancement for society. The research findings will also enhance undergraduate and graduate education with exciting examples and demonstrations of real-world systems at the intersection of distributed computing, cloud computing, and deep learning.The project will develop a novel serverless computing platform with a hybrid CPU/GPU architecture that will provide DL applications with native GPU performance. Two core components constitute the hybrid serverless computing architecture, a shim virtualized GPU (vGPU) layer and a refactored container subsystem. The shim vGPU layer enables high-performance GPU sharing for concurrent serverless functions with low latency and high scalability. This layer provides fine-grained GPU resource provisioning and performance isolation by intercepting GPU calls from serverless functions using API remoting techniques. The vGPU layer optimizes GPU performance in serverless computing via GPU context caching and locality-aware scheduling to mitigate cold-starts and unnecessary data movement. The container subsystem accelerates the entire DL lifecycle by exploiting DL model structures and pipelined model loading to parallelize CPU-to-GPU memory copy and model execution. The subsystem exploits model partitioning techniques to accelerate the hybrid CPU/GPU architecture by dynamically distributing the DL model partitions to CPU and GPU. The scientific knowledge and tools designed and implemented from this research project will provide and enable innovations for next-generation cloud computing and deep learning.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该奖项的全部或部分资金来源于《2021 年美国救援计划法案》（公法 117-2）。下一代无服务器云计算为开发人员提供了对服务器管理和管理的简化访问，包括事件驱动执行、精细化粒度资源配置、自动扩展和按量付费计费。机器学习社区正在利用无服务器云计算的这些优势来简化深度学习 (DL) 应用程序的开发和部署。然而，现有的无服务器计算平台缺乏对GPU的有效支持，阻碍了深度学习从业者利用无服务器计算进行大规模应用。该项目将开发一个具有混合CPU/GPU架构的高效无服务器计算平台，以加速DL应用程序的开发和部署。其目标是推进深度学习和无服务器计算领域的前沿方法，这将带来重大飞跃，使深度学习从业者、深度学习用户和云计算基础设施提供商受益，为社会的科学进步做出贡献。研究结果还将通过分布式计算、云计算和深度学习交叉领域的真实世界系统的令人兴奋的示例和演示来加强本科生和研究生教育。该项目将开发一种具有混合 CPU/GPU 架构的新型无服务器计算平台这将为深度学习应用程序提供原生 GPU 性能。混合无服务器计算架构由两个核心组件组成：shim 虚拟化 GPU (vGPU) 层和重构的容器子系统。 shim vGPU 层支持并发无服务器功能的高性能 GPU 共享，具有低延迟和高可扩展性。该层通过使用 API 远程处理技术拦截来自无服务器函数的 GPU 调用，提供细粒度的 GPU 资源配置和性能隔离。 vGPU 层通过 GPU 上下文缓存和位置感知调度来优化无服务器计算中的 GPU 性能，以减少冷启动和不必要的数据移动。容器子系统通过利用深度学习模型结构和流水线模型加载来并行化 CPU 到 GPU 内存复制和模型执行，从而加速整个深度学习生命周期。该子系统利用模型分区技术，通过动态地将深度学习模型分区分配给 CPU 和 GPU 来加速混合 CPU/GPU 架构。该研究项目设计和实施的科学知识和工具将为下一代云计算和深度学习提供并实现创新。该奖项反映了 NSF 的法定使命，并通过利用基金会的智力优势和更广泛的影响进行评估，认为值得支持审查标准。

项目成果

期刊论文数量（2）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

Libra: Harvesting Idle Resources Safely and Timely in Serverless Clusters

DOI：
10.1145/3588195.3592996
发表时间：
2023-08-07
期刊：
Proceedings of the 32nd International Symposium on High-Performance Parallel and Distributed Computing
影响因子：
0
作者：
Hanfei Yu;Christian Fontenot;Hao Wang;Jian Li;Xu Yuan;Seung
通讯作者：
Seung

Accelerating Serverless Computing by Harvesting Idle Resources

通过收集闲置资源加速无服务器计算