SPX: Secure, Highly-Parallel Training of Deep Neural Networks in the Cloud Using General-Purpose Shared-Memory Platforms

SPX：使用通用共享内存平台在云中对深度神经网络进行安全、高度并行的训练

• 批准号: 1725734
• 负责人: Josep Torrellas
• 金额: $ 50万
• 依托单位: University of Illinois at Urbana-Champaign
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-09-01 至 2020-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1725734&HistoricalAwards=false
• 关键词: SPX; Secure; Highly; Parallel; Training

Society is beginning to witness an explosion in the use of Deep Neural Networks (DNNs), with major impacts on many facets of human life, including health, finances, family life, and entertainment. To train DNNs, practitioners have preferred to use GPUs and, recently, specialized hardware accelerators.  Despite constituting the bulk of a data center?s compute resources, general-purpose shared-memory multiprocessors have been regarded as unattractive platforms. In this project, the Principal Investigators (PIs) think that these platforms have high potential. Consequently, this project will develop new techniques to dramatically improve shared-memory multiprocessor performance in training DNNs.  Already, shared-memory servers are compelling for several reasons: they can support a high-degree of parallelism, are general-purpose and easy to program, and provide flexible, fine-grain inter-core communication.  However, efficiently using shared-memory servers to train DNNs imposes significant challenges. First, fine-grain synchronization is still expensive, and latencies are non-trivial. In addition, when DNN training moves to an environment with multiple users sharing the same physical shared-memory platform in the cloud, privacy and integrity become major concerns.To overcome these challenges, this project will synergistically address architecture and security issues.  On the architecture side, it will augment a highly-parallel shared-memory server with support for synchronization, data movement, data sharing, and DNN sparsity structuring.  On the security side, it will investigate how shared-memory servers create novel privacy and integrity threats (for example, leaking the DNN?s sparse structure and forcing incorrect model generation), and how to defend against those threats.  The project?s broader impact is to help enable ?neural network training for everyone,? by making a ubiquitous and easy-to-program platform a viable and safe target for running these important, emerging workloads.

社会开始见证深度神经网络 (DNN) 使用的爆炸式增长，这对人类生活的许多方面产生了重大影响，包括健康、财务、家庭生活和娱乐。为了训练 DNN，从业者更喜欢使用 GPU 和 GPU。最近，尽管专用硬件加速器构成了数据中心的大部分计算资源，但通用共享内存多处理器仍被认为是最重要的。在这个项目中，首席研究员（PI）认为这些平台具有很高的潜力，原因有几个：它们可以支持高度并行性，通用且易于编程，并提供灵活、精细的平台。然而，有效地利用颗粒间的通信。首先，细粒度同步仍然很昂贵，而且当 DNN 训练转移到多个用户共享相同物理共享内存平台的环境时，延迟也很严重。为了克服这些挑战，该项目将协同解决架构和安全问题，在架构方面，它将增强高度并行。支持同步、数据移动、数据共享和 DNN 稀疏结构的共享内存服务器在安全方面，它将研究共享内存服务器如何造成新的隐私和完整性威胁（例如，泄露 DNN 的稀疏结构）。并强制生成错误的模型），以及如何防御这些威胁？该项目的广播影响是通过使每个人都能进行神经网络训练。无处不在且易于编程的平台是运行这些重要的新兴工作负载的可行且安全的目标。

项目成果

SparseTrain: Leveraging Dynamic Sparsity in Software for Training DNNs on General-Purpose SIMD Processors

• DOI: 10.1145/3410463.3414655
• 发表时间: 2019-11
• 期刊: Proceedings of the ACM International Conference on Parallel Architectures and Compilation Techniques
• 作者: Zhangxiaowen Gong;Houxiang Ji;Christopher W. Fletcher;C. Hughes;J. Torrellas

Game of Threads: Enabling Asynchronous Poisoning Attacks

• DOI: 10.1145/3373376.3378462
• 发表时间: 2020-03
• 期刊: Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems
• 作者: Jose Rodrigo Sanchez Vicarte;Benjamin Schreiber;Riccardo Paccagnella;Christopher W. Fletcher

SAVE: Sparsity-Aware Vector Engine for Accelerating DNN Training and Inference on CPUs

• DOI: 10.1109/micro50266.2020.00070
• 发表时间: 2020-10
• 期刊: 2020 53rd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO)
• 作者: Zhangxiaowen Gong;Houxiang Ji;Christopher W. Fletcher;C. Hughes;Sara S. Baghsorkhi;J. Torrellas

MicroScope: enabling microarchitectural replay attacks

MicroScope：启用微架构重放攻击

• DOI: 10.1145/3307650.3322228
• 发表时间: 2019
• 期刊: International Symposium on Computer Architecture
• 作者: Skarlatos, Dimitrios;Yan, Mengjia;Gopireddy, Bhargava;Sprabery, Read;Torrellas, Josep;Fletcher, Christopher W.
• 通讯作者: Fletcher, Christopher W.

Cache Telepathy: Leveraging Shared Resource Attacks to Learn DNN Architectures

• 发表时间: 2018-08
• 期刊: ArXiv
• 作者: Mengjia Yan;Christopher W. Fletcher;J. Torrellas