Collaborative Research: CNS Core: Medium: Exploiting Synergies Between Machine-Learning Algorithms and Hardware Heterogeneity for High-Performance and Reliable Manycore Computing

合作研究：CNS Core：Medium：利用机器学习算法和硬件异构性之间的协同作用实现高性能和可靠的众核计算

• 批准号: 1955196
• 负责人: Hai Li
• 金额: $ 45万
• 依托单位: Duke University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-06-15 至 2024-05-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1955196&HistoricalAwards=false
• 关键词: Collaborative; Research; CNS; Core; Medium

Advanced computing systems have long been enablers for breakthroughs in science, engineering, and new technologies. However, with the slowing down of Moore’s law and the relentless needs of Big-Data applications, e.g., deep learning, graph analytics, and scientific simulations, current solutions are not adequate. There is a need for innovative computer architectures and computationally efficient methods to design application-specific hardware systems to optimize performance, power consumption, and reliability. The main focus of this work is design and demonstration of a heterogeneous single-chip manycore platform, integrating CPU, GPU, accelerator, and memory cores, via a network-on-chip to avoid expensive off-chip data transfers. The goal of this project is to address the design of application-specific heterogeneous manycore systems that are poised to achieve unprecedented levels of performance and energy-efficiency for Big-Data applications. The PIs will disseminate research outcomes via publications, seminars, tutorials, and workshops. The project is also leading to the development of an interdisciplinary research-based curriculum integrating computer architectures, machine learning, and data-driven design optimization. Undergraduate and graduate students involved in this research will be trained to apply classroom knowledge to research problems that require next-generation hardware, software, and theoretical expertise. The project will lay the foundations for a novel computing paradigm for Big-Data applications that allows us to quickly design and autonomously self-manage heterogeneous manycore computing systems to improve performance, reduce power consumption, and enhance reliability. In-memory processing can overcome the memory wall, but it introduces new challenges in overall application-specific system optimization. The specific research tasks include: 1) Data-driven multi-objective design space exploration and optimization algorithms for heterogeneous manycore architectures; 2) Reliability assessment and system design for reliability; 3) Structured learning framework for autonomous resource management; and 4) Performance, power, and reliability evaluation using emerging Big-Data application workloads. This framework will combine the benefits of multi-objective design space exploration and optimization, heterogeneity in computation and communication, and data-driven algorithms to improve performance, energy-efficiency, and reliability of manycore platforms.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

长期以来，先进的计算系统一直是科学，工程和新技术突破的推动力。但是，随着摩尔定律的放缓以及大数据应用程序的无情需求，例如深度学习，图形分析和科学模拟，当前的解决方案不足。需要创新的计算机体系结构和计算有效的方法来设计特定于应用程序的硬件系统，以优化性能，功耗和可靠性。这项工作的主要重点是设计和演示异质的单芯片多核平台，通过网络芯片进行集成CPU，GPU，加速器和内存核心，以避免使用昂贵的芯片外数据传输。该项目的目的是解决特定应用的异质多核系统的设计，这些系统被中毒以达到前所未有的绩效和能源效率的大数据应用程序。 PI将通过出版物，半手，教程和研讨会来传播研究成果。该项目还导致开发基于跨学科的基于研究的课程，该课程整合了计算机架构，机器学习和数据驱动的设计优化。参与这项研究的本科生和研究生将接受培训，以将课堂知识应用于需要下一代硬件，软件和理论专业知识的研究问题。该项目将为大数据应用程序的新型计算范式奠定基础，这使我们能够快速设计和自主自主自我管理的许多核心计算系统，以提高性能，降低功耗和增强可靠性。内存处理可以克服内存墙，但它引入了整体应用系统优化的新挑战。特定的研究任务包括：1）数据驱动的多目标设计空间探索和优化算法，用于异质多核体系结构； 2）可靠性评估和系统设计； 3）自主资源管理的结构化学习框架； 4）使用新兴的大数据应用程序工作负载进行性能，功率和可靠性评估。该框架将结合多目标设计空间探索和优化的好处，计算和沟通中的异质性以及数据驱动的算法，以提高绩效，能源效率和许多核平台的可靠性。该奖项反映了NSF的法规任务，并通过评估通过基金会的知识优点和广泛的criitia crietia crietia crietia crietia criter criter crietia crietia criitia crietia criitia crietia crietia crietia crietia crietia crietia crietia crietia cripitia cripitia cripitia cripitia均被认为是宝贵的。

项目成果

Approximate Computing and the Efficient Machine Learning Expedition

• DOI: 10.1145/3508352.3561105
• 发表时间: 2022-10
• 期刊: 2022 IEEE/ACM International Conference On Computer Aided Design (ICCAD)
• 作者: J. Henkel;Hai Helen Li;A. Raghunathan;M. Tahoori;Swagath Venkataramani;Xiaoxuan Yang;Georgios Zervakis

High-Throughput Training of Deep CNNs on ReRAM-Based Heterogeneous Architectures via Optimized Normalization Layers

通过优化的归一化层在基于 ReRAM 的异构架构上进行深度 CNN 的高吞吐量训练

• DOI: 10.1109/tcad.2021.3083684
• 发表时间: 2022
• 期刊: IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
• 影响因子: 2.9
• 作者: Joardar, Biresh Kumar;Deshwal, Aryan;Doppa, Janardhan Rao;Pande, Partha Pratim;Chakrabarty, Krishnendu
• 通讯作者: Chakrabarty, Krishnendu

DARe: DropLayer-Aware Manycore ReRAM architecture for Training Graph Neural Networks

• DOI: 10.1109/iccad51958.2021.9643511
• 发表时间: 2021-11
• 期刊: 2021 IEEE/ACM International Conference On Computer Aided Design (ICCAD)
• 作者: Aqeeb Iqbal Arka;B. K. Joardar;J. Doppa;P. Pande;K. Chakrabarty

ReTransformer: ReRAM-based Processing-in-Memory Architecture for Transformer Acceleration

• DOI: 10.1145/3400302.3415640
• 发表时间: 2020-11
• 期刊: 2020 IEEE/ACM International Conference On Computer Aided Design (ICCAD)
• 作者: Xiaoxuan Yang;Bonan Yan;H. Li;Yiran Chen

ReaLPrune: ReRAM Crossbar-Aware Lottery Ticket Pruning for CNNs

• DOI: 10.1109/tetc.2022.3223630
• 发表时间: 2023-04
• 期刊: IEEE Transactions on Emerging Topics in Computing
• 影响因子: 5.9
• 作者: B. K. Joardar;J. Doppa;Hai Helen Li;K. Chakrabarty;P. Pande