SHF: Small: Algorithms and Software for Scalable Kernel Methods

SHF：小型：可扩展核方法的算法和软件

基本信息

批准号：
1817048
负责人：
George Biros
金额：
$ 47.62万
依托单位：
University of Texas at Austin
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2018
资助国家：
美国
起止时间：
2018-07-01 至 2021-06-30
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1817048&HistoricalAwards=false
关键词：
SHF Small Algorithms Software Scalable

项目摘要

Scientists and engineers are increasingly interested in using machine learning methods on huge datasets that cannot be processed on a single workstation. At the same time public and private institutions are making significant investments on high-performance computing (HPC) clusters equipped with thousands of leading edge processors and network connectivity. However, despite the availability of such HPC systems, data analysis tasks are mostly restricted to a single or a few workstations. The reason is that, with few exceptions, existing machine learning software does not scale efficiently on HPC systems. The need to process in-situ large scientific and engineering datasets is not met with current software and significant downsampling is required in order to use existing tools. A serious bottleneck in current artificial intelligence (AI) workflows is the significant cost of training for large scale problems. The slow convergence of existing methods and the large number of calibration hyper-parameters (learning rate, batch size, and other knobs that control the performance of the AI system) make training extremely expensive. Design and analysis of scalable optimization algorithms for faster training, that is the fitting of the machine learning (ML) model parameters to the data, are needed for analytics in real time and at scale, which is the goal of this project.The proposed research will introduce novel numerical methods and parallel algorithms for second-order/Newton methods that will be tailored to machine learning (ML) models and will be many orders of magnitude faster than the existing state of-the-art (first-order methods like steepest descent). The researchers plan to design, analyze, and implement robust approximations for covariance matrices, a class of matrices in AI and computational statistics, used in statistical analysis (e.g., sampling, risk assessment, and uncertainty quantification). The investigators plan to design, analyze, and implement scalable fast algorithms in the context of high-performance computing for the so called nearest-neighbor problem, a particular method in ML, data analysis, and information retrieval. The resulting software library will provide a means for end-to-end tools for discovery and innovation and provide new capabilities in the NSF XSEDE infrastructure project. Along with research activities, an educational and dissemination program is designed to communicate the results of this work to both students and researchers, as well as a more general audience of computational and application scientists.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

科学家和工程师越来越有兴趣在无法在单个工作站上处理的大型数据集上使用机器学习方法。同时，公共和私人机构正在对高性能计算（HPC）群集进行大量投资，配备了数千个前沿处理器和网络连接。但是，尽管有此类HPC系统的可用性，但数据分析任务主要仅限于单个或几个工作站。原因是，除了少数例外，现有的机器学习软件不会在HPC系统上有效扩展。当前的软件不满足现场的大型科学和工程数据集的需求，并且需要大量的缩放采样才能使用现有工具。当前人工智能（AI）工作流程中严重的瓶颈是针对大规模问题的培训的重要成本。现有方法和大量校准超参数（学习率，批次大小和其他控制AI系统性能的旋钮）的缓慢收敛性使训练非常昂贵。实时和大规模分析需要进行更快培训的可扩展优化算法的设计和分析，即机器学习（ML）模型参数的拟合（ML）模型参数，这是该项目的目标。拟议的研究将介绍新的数值方法和型号的型号和机器的模型（MANDER MALINDER）（MANDER MANDER）（MARNEVER）的新数值方法和平行算法（MANDER MANICEL）（MARNEVER）。（例如最陡峭的下降）（如一阶方法）。研究人员计划针对协方差矩阵设计，分析和实施强大的近似值，即AI和计算统计中的一类矩阵，用于统计分析（例如，采样，风险评估和不确定性量化）。研究人员计划在所谓的最近邻居问题（ML，数据分析和信息检索中）设计，分析和实施可扩展的快速算法。最终的软件库将为发现和创新的端到端工具提供一种手段，并在NSF XSEDE基础架构项目中提供新的功能。除研究活动外，一项教育和传播计划旨在将这项工作的结果传达给学生和研究人员，以及更普遍的计算和应用科学家受众。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的知识分子优点和更广泛影响的审查标准来通过评估来支持的。

项目成果

期刊论文数量（6）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

Distributed O(N) Linear Solver for Dense Symmetric Hierarchical Semi-Separable Matrices

密集对称分层半可分离矩阵的分布式 O(N) 线性求解器

DOI：
10.1109/mcsoc.2019.00008
发表时间：
2019
期刊：
2019 IEEE 13th International Symposium on Embedded Multicore/Many-core Systems-on-Chip (MCSoC
影响因子：
0
作者：
Yu, Chenhan D.;Reiz, Severin;Biros, George
通讯作者：
Biros, George

Hardware Accelerator Integration Tradeoffs for High-Performance Computing: A Case Study of GEMM Acceleration in N-Body Methods

高性能计算的硬件加速器集成权衡：N 体方法中 GEMM 加速的案例研究

DOI：
10.1109/tpds.2021.3056045
发表时间：
2021
期刊：
IEEE Transactions on Parallel and Distributed Systems
影响因子：
5.3
作者：
Asri, Mochamad;Malhotra, Dhairya;Wang, Jiajun;Biros, George;John, Lizy K;Gerstlauer, Andreas
通讯作者：
Gerstlauer, Andreas

RCHOL: Randomized Cholesky Factorization for Solving SDD Linear Systems

RCHOL：用于求解 SDD 线性系统的随机 Cholesky 分解

DOI：
10.1137/20m1380624
发表时间：
2021
期刊：
SIAM Journal on Scientific Computing
影响因子：
3.1
作者：
Chen, Chao;Liang, Tianyu;Biros, George
通讯作者：
Biros, George

ANODE: Unconditionally Accurate Memory-Efficient Gradients for Neural ODEs

DOI：
10.24963/ijcai.2019/103
发表时间：
2019-02
期刊：
ArXiv
影响因子：
0
作者：
A. Gholami;K. Keutzer;G. Biros
通讯作者：
A. Gholami;K. Keutzer;G. Biros

ANODEV2: A Coupled Neural ODE Framework

DOI：
发表时间：
2019
期刊：
ArXiv
影响因子：
0
作者：
Tianjun Zhang;Z. Yao;A. Gholami;Joseph Gonzalez;K. Keutzer;Michael W. Mahoney;G. Biros
通讯作者：
Tianjun Zhang;Z. Yao;A. Gholami;Joseph Gonzalez;K. Keutzer;Michael W. Mahoney;G. Biros