Software optimization by synergy of machine learning and high performance computing

机器学习和高性能计算协同优化软件

基本信息

批准号：
18F18786
负责人：
須田礼仁
金额：
$ 0.96万
依托单位：
The University of Tokyo
依托单位国家：
日本
项目类别：
Grant-in-Aid for JSPS Fellows
财政年份：
2018
资助国家：
日本
起止时间：
2018-11-09 至 2021-03-31
项目状态：
已结题

来源：
https://kaken.nii.ac.jp/grant/KAKENHI-PROJECT-18F18786/
关键词：
通信削減アルゴリズム matrix powers kernel parallel computing sparse matrix spMV iterative methods 深層学習 Dense layer Pruning データ圧縮

项目摘要

We implemented DMPK (Diamond Matrix Powers Kernel) with parallelization of MPI and optimized assignment of tasks to processors. We also analyzed the amount of communication and redundant computation when using different number of phases and compared them to PA1 and PA2, which are the known methods of matrix powers kernel. These results been presented at HPCAsia2020 in Fukuoka.Matrix Powers Kernel (MPK) algorithms calculate the vector Akx, obtained by multiplying an initial vector x with the k-th power of matrix A. Our algorithm, Diamond Matrix Powers Kernel (DMPK) generalizes the MPK algorithms PA1 and PA2 by Demmel et al. PA1 and PA2 can be used for general matrices. They improve performance by reducing the amount of communication, which is often the bottleneck, but they introduce redundant computations. In scientific computations with regular access patterns, diamond tiling algorithms achieve similar communication avoidance without introducing any redundant communication by introducing moving index domains. By combining these two approaches, DMPK, is applicable to general matrices and makes it possible to reduce the amount of redundant computation at the price of slightly higher amount of communication. This is done by translating the concept of moving index domains to general matrices: the algorithm is performed in “phases” and after each phase the graph (corresponding to the matrix) is repartitioned.

我们实现了DMPK（Diamond Matrix Powers内核），并通过MPI并行化，并优化了任务分配给处理器。我们还分析了使用不同数量的阶段时的通信和冗余计算量，并将其与PA1和PA2进行了比较，PA1和PA2是矩阵幂核的已知方法。这些结果在fukuoka的HPCASIA2020中介绍。MatrixPowers内核（MPK）算法计算矢量AKX，通过将初始矢量X与矩阵A的k-The乘以k-th的功率A算法A.我们的Algorithm，我们的diamond Matrix Matrix pabers kernel（dmpk）Algrith angm and angm and angm and Mpk Mpk Mpk Mpk Mpk Mpk and Matrix A. PA1和PA2可用于通用矩阵。它们通过减少沟通量来提高性能，这通常是瓶颈，但它们引入了冗余计算。在具有常规访问模式的科学计算中，钻石瓷砖算法可以通过引入移动索引域来实现类似的沟通避免，而无需引入任何冗余沟通。通过将这两种方法结合在一起，即DMPK，适用于通用物质，并可以以稍高的通信量减少冗余计算量。这是通过将移动索引域域的概念转换为通用物质的概念来完成的：该算法是在“阶段”中执行的，并且在每个阶段之后，图表（对应于矩阵）将重新分配。