Collaborative Research: EMBRACE: Evolvable Methods for Benchmarking Realism through Application and Community Engagement

合作研究：拥抱：通过应用和社区参与对现实主义进行基准测试的演化方法

• 批准号: 1535025
• 负责人: Jack Dongarra
• 金额: $ 12.5万
• 依托单位: University of Tennessee Knoxville
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2015
• 资助国家: 美国
• 起止时间: 2015-09-01 至 2017-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1535025&HistoricalAwards=false
• 关键词: Collaborative; Research; EMBRACE; Evolvable; Methods; Collaborative; Research; EMBRACE; Evolvable; Methods

High-performance computing opens new avenues in science, planning, health, and defense that are not economically achievable by pure experimentation. New progress areas require new evaluation methods, packaged in benchmarks, for selecting computational hardware and systems. EMBRACE will guide selection of computing technologies for different scientific areas with different needs. Current approaches assume that one kind of platform addresses all problems, but growing use in bioinformatics, urban planning, medicine, and more fields have demonstrated new, uncovered system requirements. EMBRACE opens the process through community competitions. These competitions will both build benchmarks useful for multiple scientific areas as well as build a community around benchmarking science. The evolving measurements will help scientists best match their applications with computing platforms.Despite the long-standing interest and important impacts of benchmarking, there is no resolution on the most rational way to design, develop, and evolve forward-looking benchmarks, and to correctly interpret their results. Over time, the community has broadened, the platforms have changed dramatically, and the applications have evolved, yet today's benchmarks have not necessarily evolved with them. The primary objective of this proposal is to sustainably advance the science of benchmarking based on current and future needs. A second objective is to create sustainable incentives that drive research and innovation. A third objective is to produce a select set of the most useful and evolving benchmarks, benchmarking methods, or benchmarking analyses that can be accepted by the community. The vision and objectives will be realized by establishing a new, sustainable, and community-driven competition, the Evolvable Methods for Benchmarking Realism through Application and Community Engagement (EMBRACE) Workshop, in which teams compete in categories directly relevant to performance measurement and benchmarking. Submissions to the contest will include code, technical papers and presentations. In consultation with leaders in various application domains and with vendor representatives, the EMBRACE team will be responsible for defining the contest categories, framing the rules, recruiting both the participants and the judges, and monitoring and managing the process as it takes place. It will be an open forum to advance the broader scientific community's understanding of fundamental questions in the area of benchmarking.

高性能计算开辟了科学，规划，健康和防御方面的新途径，这些途径在纯粹的实验上是无法经济实现的。新的进度领域需要新的评估方法，以基准为基准，以选择计算硬件和系统。拥抱将指导针对有不同需求的不同科学领域的计算技术选择。当前的方法假设一种平台解决了所有问题，但是在生物信息学，城市规划，医学和更多领域中的使用越来越多，已经证明了新的，未发现的系统要求。拥抱通过社区竞争打开过程。这些竞赛都将建立对多个科学领域有用的基准，并围绕基准科学建立社区。不断发展的测量结果将有助于科学家与计算平台最好地匹配他们的应用程序。尽管如此，基准测试的长期兴趣和重要影响，但对最合理的设计，开发和发展前瞻性基准的方法没有解决方案，无法正确解释其结果。随着时间的流逝，社区扩大了，平台已经发生了巨大变化，并且应用程序已经发展，但是今天的基准不一定随着它们而发展。该提案的主要目的是根据当前和未来的需求可持续发展基准测试的科学。第二个目标是创造可持续的激励措施，以推动研究和创新。第三个目标是生产一组最有用，最有用的基准测试，基准方法或社区可以接受的基准分析。通过建立新的，可持续和社区驱动的竞争，可以实现愿景和目标，这是通过应用和社区参与（拥抱）研讨会来基准现实主义的可发展方法，在该方法中，团队在与绩效测量和基准测试直接相关的类别中竞争。比赛的提交将包括代码，技术论文和演示文稿。在与各种应用领域和供应商代表中的领导人协商时，Embrace团队将负责定义比赛类别，制定规则，招募参与者和法官，并在此过程中监视和管理和管理该过程。这将是一个开放的论坛，以提高更广泛的科学界对基准测试领域的基本问题的理解。