Collaborative Research: SHF:SMALL: Compile-Parallelize-Schedule-Retarget-Repeat (EASER) Paradigm for Dealing with Extreme Heterogeneity

合作研究：SHF:SMALL：处理极端异构性的编译-并行化-调度-重定向-重复 (EASER) 范式

• 批准号: 2146852
• 负责人: Gagan Agrawal
• 金额: $ 25万
• 依托单位: AUGUSTA UNIVERSITY RESEARCH INSTITUTE, INC.
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-06-15 至 2023-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2146852&HistoricalAwards=false
• 关键词: Collaborative; Research; SHF; SMALL; Compile

Heterogeneity in computing refers to having a variety of devices present within one computing system or even within one node of a cluster. A number of technological trends are making a high degree of heterogeneity inevitable in High Performance Computing (HPC), leading to research along many directions. The traditional scheduling problem, which refers to taking a set of programs to be executed and mapping them to the available resources, becomes more complicated in the presence of such heterogeneity, as the schedulers need to interact with the compiler also. The goal of this project is to consider new paradigms for application execution in view of these developments and conduct research in developing predictions of execution times, compilation, parallelization, and scheduling. Traditionally, deciding (likely manually) how an application is to be parallelized, compilation, and cluster-level scheduling are done sequentially and independently. The investigators posit that their isolated treatment is not going to be acceptable when one tries to optimize for multi-tenant heterogeneous clusters. Instead, the investigators envision a requirement that can be referred to as EASER -- compilE-pArallelize-Schedule-rEtarget-Repeat. To elaborate on the vision, in the EASER paradigm the compiler first maps the core functions to a specific device, generating predictions of execution time that are input to the parallelization approach selection module, and together they produce a final executable. Subsequently, this binary is presented to the scheduler, which assesses the job queue and might suggest alternative configuration(s)/device(s). If so, a retargeting module is to be invoked, leading to a potential repetition of the above steps. This project develops, supports, and evaluates the EASER framework in the context of a cluster that executes emerging machine learning (ML) workloads. Research is proposed in the following areas: 1) Compiler-Driven Performance Prediction -- It includes a novel strategy that comprises a general model for predicting SIMD/VLIW performance and an operator classification based approach to developing a memory hierarchy performance model. 2) Integrated Job Scheduling and Parallelization Strategy Selection -- Building on the performance prediction models, these two (conventionally independent) modules are integrated, by including parameterized and incremental parallelization strategy selection methods and aggressively reducing the search space in scheduling methods. 3) Retargeting Compiler -- By classifying optimizations as either architecture-dependent or independent, a retargeting compiler for ML workloads will be developed. This project will also make several contributions to education and human resource development. Both investigators will be introducing course(s) (material) at the intersection of computer systems and machine learning, bringing attention to ML-related workloads in computer systems education. A majority of funds at each University will be used to support Ph.D. students in their research, who will be trained to work across traditional (sub-) areas. Both investigators are strongly committed to increasing diversity in computing fields and have a strong record of supervising members of underrepresented groups in their research programs. Building on their Universities' existing connections, they will be further working on improving diversity at all levels.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

计算中的异质性是指在一个计算系统中甚至在集群的一个节点内具有各种设备。许多技术趋势在高性能计算（HPC）中不可避免地具有高度的异质性，从而沿着许多方向进行了研究。 传统的调度问题是指在存在此类异质性的情况下，要执行一组程序并将其映射到可用资源中，因为调度程序也需要与编译器进行交互。鉴于这些发展，该项目的目的是考虑用于应用程序执行的新范式，并在开发执行时间，汇编，并行化和调度的预测时进行研究。 传统上，（可能是手动）如何并行编译和集群级调度（可能是手动）进行的。研究人员认为，当人们试图优化多租户异质簇时，他们的孤立处理将无法接受。相反，调查人员设想了一个可以称为更宽松的要求 - 汇编 - 划分 - 安排retarget-repeat。为了详细说明视觉，在较为范式中，编译器首先将核心功能映射到特定的设备，从而生成对并行化方法选择模块输入的执行时间的预测，并共同产生最终的可执行文件。随后，将此二进制呈现给调度程序，该调度程序评估了作业队列，并可能建议替代配置（S）/设备。如果是这样，则将调用重新定位模块，从而导致上述步骤的潜在重复。 该项目在执行新兴计算机学习（ML）工作量的集群的上下文中开发，支持和评估较较高的框架。在以下领域提出了研究：1）编译器驱动的性能预测 - 它包括一种新型策略，该策略包括一个通用模型，用于预测SIMD/VLIW性能和基于操作员分类的方法，用于开发内存层次结构绩效模型。 2）集成的作业计划和并行化策略选择 - 基于绩效预测模型，通过包括参数化和增量并行化策略选择方法并积极地减少调度方法中的搜索空间，从而集成了这两个（常规独立的）模块。 3）重新定位编译器 - 通过将优化分类为体系结构依赖性或独立，将开发用于ML工作负载的重新定位编译器。 该项目还将为教育和人力资源开发做出一些贡献。两位研究人员都将在计算机系统和机器学习的交集中引入课程（S）（材料），从而引起对计算机系统教育中与ML相关的工作量的关注。每所大学的大多数资金将用于支持博士学位。他们的研究中的学生将接受培训，可以在传统（子）领域工作。两位研究人员都强烈致力于提高计算领域的多样性，并在其研究计划中拥有监督代表性不足的群体成员的记录。在大学的现有联系的基础上，他们将进一步努力改善各个级别的多样性。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的知识分子优点和更广泛的影响审查标准来评估的。

项目成果

GPU Adaptive In-situ Parallel Analytics (GAP)

• DOI: 10.1145/3559009.3569661
• 发表时间: 2022-01-01
• 期刊: PROCEEDINGS OF THE 2022 31ST INTERNATIONAL CONFERENCE ON PARALLEL ARCHITECTURES AND COMPILATION TECHNIQUES, PACT 2022
• 作者: Xing，Haoyuan;Agrawal，Gagan;Ramnath，Rajiv
• 通讯作者: Ramnath，Rajiv

End-to-End LU Factorization of Large Matrices on GPUs

• DOI: 10.1145/3572848.3577486
• 发表时间: 2023-02
• 期刊: Proceedings of the 28th ACM SIGPLAN Annual Symposium on Principles and Practice of Parallel Programming
• 作者: Yang Xia;Peng Jiang;G. Agrawal;R. Ramnath