SHF: Small: Scalable Trace-Based Tools for In-Situ Data Analysis of HPC Applications (ScalaJack)

SHF：小型：用于 HPC 应用程序现场数据分析的可扩展的基于跟踪的工具 (ScalaJack)

基本信息

批准号：
1217748
负责人：
Frank Mueller
金额：
$ 45.74万
依托单位：
North Carolina State University
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2012
资助国家：
美国
起止时间：
2012-06-01 至 2017-05-31
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1217748&HistoricalAwards=false
关键词：
SHF Small Scalable Trace Based

项目摘要

Production codes on supercomputers are struggling to remain scalableeach time the processor core count increases by a factor of 10, eventhough they run efficiently at smaller scale.But root cause diagnosis fails at petascale since (1) symptoms ofperformance problems can be subtle, (2) only fewmetrics can be efficiently collected and (3) tools can only feasibly recorda small subset of even these metrics.This work addresses these problems by creating a framework that allowsapplication developers to focus on data analysis that drives customizeddata extraction combined with on-the-fly analysis specifically gearedto their individual problems. This is accomplished by combining traceanalysis and in-situ data analysis techniques at runtime, therebylifting data reduction to a new level where it IS analysis. With thisapproach, modular measurement and analysis components are combined toselectively extract representative data from production codes in aproblem-specific manner, which enables root cause analysis.The work demonstrates the feasibility of customized dataextraction and analysis at scale for root cause analysis on currentand forthcoming multi-petascale supercomputers. It thus contributesto sustain scalable scientific computing into the future up to the largestscales. Results of this work will be contributed as open-source codeto the research community and beyond as done, allowing other groups tonot only build tools on top of our framework but also contribute theirown components.

每当处理器核心数量增加 10 倍时，超级计算机上的生产代码就很难保持可扩展性，尽管它们在较小的规模上运行效率很高。但根本原因诊断在千万亿级规模上失败，因为 (1) 性能问题的症状可能很微妙，(2)只能有效地收集很少的指标，并且 (3) 工具甚至只能记录这些指标的一小部分。这项工作通过创建一个框架来解决这些问题，该框架允许应用程序开发人员专注于数据分析，驱动定制的数据提取与即时相结合专门针对他们个人问题的分析。这是通过在运行时结合跟踪分析和现场数据分析技术来实现的，从而将数据简化提升到分析的新水平。通过这种方法，模块化测量和分析组件相结合，以特定问题的方式有选择地从生产代码中提取代表性数据，从而实现根本原因分析。这项工作证明了大规模定制数据提取和分析用于当前和即将到来的多问题根本原因分析的可行性。千万亿级超级计算机。因此，它有助于维持未来最大规模的可扩展科学计算。这项工作的结果将作为开源代码贡献给研究社区及其他社区，使其他团体不仅可以在我们的框架之上构建工具，还可以贡献自己的组件。