Collaborative Research: CNS Core: Medium: Exploiting Synergies Between Machine-Learning Algorithms and Hardware Heterogeneity for High-Performance and Reliable Manycore Computing

合作研究：CNS Core：Medium：利用机器学习算法和硬件异构性之间的协同作用实现高性能和可靠的众核计算

• 批准号: 1955196
• 负责人: Hai Li
• 金额: $ 45万
• 依托单位: Duke University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-06-15 至 2024-05-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1955196&HistoricalAwards=false
• 关键词: Collaborative; Research; CNS; Core; Medium

Advanced computing systems have long been enablers for breakthroughs in science, engineering, and new technologies. However, with the slowing down of Moore’s law and the relentless needs of Big-Data applications, e.g., deep learning, graph analytics, and scientific simulations, current solutions are not adequate. There is a need for innovative computer architectures and computationally efficient methods to design application-specific hardware systems to optimize performance, power consumption, and reliability. The main focus of this work is design and demonstration of a heterogeneous single-chip manycore platform, integrating CPU, GPU, accelerator, and memory cores, via a network-on-chip to avoid expensive off-chip data transfers. The goal of this project is to address the design of application-specific heterogeneous manycore systems that are poised to achieve unprecedented levels of performance and energy-efficiency for Big-Data applications. The PIs will disseminate research outcomes via publications, seminars, tutorials, and workshops. The project is also leading to the development of an interdisciplinary research-based curriculum integrating computer architectures, machine learning, and data-driven design optimization. Undergraduate and graduate students involved in this research will be trained to apply classroom knowledge to research problems that require next-generation hardware, software, and theoretical expertise. The project will lay the foundations for a novel computing paradigm for Big-Data applications that allows us to quickly design and autonomously self-manage heterogeneous manycore computing systems to improve performance, reduce power consumption, and enhance reliability. In-memory processing can overcome the memory wall, but it introduces new challenges in overall application-specific system optimization. The specific research tasks include: 1) Data-driven multi-objective design space exploration and optimization algorithms for heterogeneous manycore architectures; 2) Reliability assessment and system design for reliability; 3) Structured learning framework for autonomous resource management; and 4) Performance, power, and reliability evaluation using emerging Big-Data application workloads. This framework will combine the benefits of multi-objective design space exploration and optimization, heterogeneity in computation and communication, and data-driven algorithms to improve performance, energy-efficiency, and reliability of manycore platforms.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

长期以来，先进的计算系统一直是科学、工程和新技术突破的推动者，然而，随着摩尔定律的放缓和大数据应用（例如深度学习、图形分析和科学模拟）的不断需求，当前的计算系统已经成为科学、工程和新技术突破的推动力。需要创新的计算机架构和计算高效的方法来设计专用硬件系统，以优化性能、功耗和可靠性。这项工作的主要重点是异构单芯片的设计和演示。众核该项目的目标是解决专用异构众核系统的设计问题，通过片上网络集成 CPU、GPU、加速器和内存核心，以避免昂贵的片外数据传输。 PI 将通过出版物、研讨会、教程和讲习班传播研究成果，以实现大数据应用的前所未有的性能和能源效率。参与这项研究的本科生和研究生将接受培训，将课堂知识应用于需要下一代硬件、软件和理论专业知识的研究问题。一种针对大数据应用的新颖计算范式，使我们能够快速设计和自主管理异构多核计算系统，以提高性能、降低功耗并增强可靠性。内存处理可以克服内存墙，但它引入了新的技术。整个特定应用系统的挑战具体研究任务包括：1）异构众核架构的数据驱动的多目标设计空间探索和优化算法；2）可靠性评估和系统设计；3）自主资源管理的结构化学习框架；使用新兴大数据应用工作负载进行性能、功耗和可靠性评估该框架将结合多目标设计空间探索和优化、计算和通信异构性以及数据驱动算法的优点，以提高性能。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Approximate Computing and the Efficient Machine Learning Expedition

• DOI: 10.1145/3508352.3561105
• 发表时间: 2022-10
• 期刊: 2022 IEEE/ACM International Conference On Computer Aided Design (ICCAD)
• 作者: J. Henkel;Hai Helen Li;A. Raghunathan;M. Tahoori;Swagath Venkataramani;Xiaoxuan Yang;Georgios Zervakis

High-Throughput Training of Deep CNNs on ReRAM-Based Heterogeneous Architectures via Optimized Normalization Layers

通过优化的归一化层在基于 ReRAM 的异构架构上进行深度 CNN 的高吞吐量训练

• DOI: 10.1109/tcad.2021.3083684
• 发表时间: 2022
• 期刊: IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
• 影响因子: 2.9
• 作者: Joardar, Biresh Kumar;Deshwal, Aryan;Doppa, Janardhan Rao;Pande, Partha Pratim;Chakrabarty, Krishnendu
• 通讯作者: Chakrabarty, Krishnendu

DARe: DropLayer-Aware Manycore ReRAM architecture for Training Graph Neural Networks

• DOI: 10.1109/iccad51958.2021.9643511
• 发表时间: 2021-11
• 期刊: 2021 IEEE/ACM International Conference On Computer Aided Design (ICCAD)
• 作者: Aqeeb Iqbal Arka;B. K. Joardar;J. Doppa;P. Pande;K. Chakrabarty

ReTransformer: ReRAM-based Processing-in-Memory Architecture for Transformer Acceleration

• DOI: 10.1145/3400302.3415640
• 发表时间: 2020-11
• 期刊: 2020 IEEE/ACM International Conference On Computer Aided Design (ICCAD)
• 作者: Xiaoxuan Yang;Bonan Yan;H. Li;Yiran Chen

Multi-Objective Optimization of ReRAM Crossbars for Robust DNN Inferencing under Stochastic Noise

• DOI: 10.1109/iccad51958.2021.9643444
• 发表时间: 2021-09
• 期刊: 2021 IEEE/ACM International Conference On Computer Aided Design (ICCAD)
• 作者: Xiaoxuan Yang;Syrine Belakaria;B. K. Joardar;Huanrui Yang;J. Doppa;P. Pande;K. Chakrabarty;Hai Li