SPX: Secure, Highly-Parallel Training of Deep Neural Networks in the Cloud Using General-Purpose Shared-Memory Platforms

SPX：使用通用共享内存平台在云中对深度神经网络进行安全、高度并行的训练

• 批准号: 1725734
• 负责人: Josep Torrellas
• 金额: $ 50万
• 依托单位: University of Illinois at Urbana-Champaign
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-09-01 至 2020-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1725734&HistoricalAwards=false
• 关键词: SPX; Secure; Highly; Parallel; Training

Society is beginning to witness an explosion in the use of Deep Neural Networks (DNNs), with major impacts on many facets of human life, including health, finances, family life, and entertainment. To train DNNs, practitioners have preferred to use GPUs and, recently, specialized hardware accelerators.  Despite constituting the bulk of a data center?s compute resources, general-purpose shared-memory multiprocessors have been regarded as unattractive platforms. In this project, the Principal Investigators (PIs) think that these platforms have high potential. Consequently, this project will develop new techniques to dramatically improve shared-memory multiprocessor performance in training DNNs.  Already, shared-memory servers are compelling for several reasons: they can support a high-degree of parallelism, are general-purpose and easy to program, and provide flexible, fine-grain inter-core communication.  However, efficiently using shared-memory servers to train DNNs imposes significant challenges. First, fine-grain synchronization is still expensive, and latencies are non-trivial. In addition, when DNN training moves to an environment with multiple users sharing the same physical shared-memory platform in the cloud, privacy and integrity become major concerns.To overcome these challenges, this project will synergistically address architecture and security issues.  On the architecture side, it will augment a highly-parallel shared-memory server with support for synchronization, data movement, data sharing, and DNN sparsity structuring.  On the security side, it will investigate how shared-memory servers create novel privacy and integrity threats (for example, leaking the DNN?s sparse structure and forcing incorrect model generation), and how to defend against those threats.  The project?s broader impact is to help enable ?neural network training for everyone,? by making a ubiquitous and easy-to-program platform a viable and safe target for running these important, emerging workloads.

社会开始见证使用深神经网络（DNN）的爆炸，对人类生活的许多方面，包括健康，财务，家庭生活和娱乐产生了重大影响。为了培训DNNS，从业者更喜欢使用GPU，并且最近使用专门的硬件加速器。尽管构成了数据中心的大部分计算资源，但通用共享内存的多处理器仍被视为无吸引力的平台。在这个项目中，主要研究人员（PIS）认为这些平台具有很高的潜力。因此，该项目将开发新的技术，以显着改善培训DNN中的共享内存多处理器性能。共享内存服务器已经引人注目的是有几个原因：它们可以支持高度的并行性，是通用的，易于编程，并提供灵活的，细粒的核心间沟通。但是，有效地使用共享内存服务器训练DNN会遇到重大挑战。首先，细粒度的同步仍然很昂贵，而延迟则不平淡。此外，当DNN培训转移到环境中，多个用户在云中共享相同的物理共享存储器平台，隐私和完整性就成为主要问题。要克服这些挑战，该项目将协同解决架构和安全问题。在体系结构方面，它将增强一台高度并行的共享内存服务器，并支持同步，数据移动，数据共享和DNN稀疏结构。在安全方面，它将调查共享的内存服务器如何创建新颖的隐私和诚信威胁（例如，泄露DNN的稀疏结构并强迫模型生成不正确），以及如何防御这些威胁。该项目的广播公司的影响是为了帮助启用每个人的神经网络培训吗？通过使无处不在且易于制图的平台成为运行这些重要的，新兴的工作负载的可行目标。

项目成果

SparseTrain: Leveraging Dynamic Sparsity in Software for Training DNNs on General-Purpose SIMD Processors

• DOI: 10.1145/3410463.3414655
• 发表时间: 2019-11
• 期刊: Proceedings of the ACM International Conference on Parallel Architectures and Compilation Techniques
• 作者: Zhangxiaowen Gong;Houxiang Ji;Christopher W. Fletcher;C. Hughes;J. Torrellas

SAVE: Sparsity-Aware Vector Engine for Accelerating DNN Training and Inference on CPUs

• DOI: 10.1109/micro50266.2020.00070
• 发表时间: 2020-10
• 期刊: 2020 53rd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO)
• 作者: Zhangxiaowen Gong;Houxiang Ji;Christopher W. Fletcher;C. Hughes;Sara S. Baghsorkhi;J. Torrellas

Game of Threads: Enabling Asynchronous Poisoning Attacks

• DOI: 10.1145/3373376.3378462
• 发表时间: 2020-03
• 期刊: Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems
• 作者: Jose Rodrigo Sanchez Vicarte;Benjamin Schreiber;Riccardo Paccagnella;Christopher W. Fletcher

Cache Telepathy: Leveraging Shared Resource Attacks to Learn DNN Architectures

• 发表时间: 2018-08
• 期刊: ArXiv
• 作者: Mengjia Yan;Christopher W. Fletcher;J. Torrellas

MicroScope: enabling microarchitectural replay attacks

MicroScope：启用微架构重放攻击

• DOI: 10.1145/3307650.3322228
• 发表时间: 2019
• 期刊: International Symposium on Computer Architecture
• 作者: Skarlatos, Dimitrios;Yan, Mengjia;Gopireddy, Bhargava;Sprabery, Read;Torrellas, Josep;Fletcher, Christopher W.
• 通讯作者: Fletcher, Christopher W.