EAGER: Preserve/Destroy Decisions for Simulation Data in Computational Physics and Beyond

EAGER：计算物理及其他领域模拟数据的保存/销毁决策

• 批准号: 1839010
• 负责人: Victoria Stodden
• 金额: $ 30万
• 依托单位: University of Illinois at Urbana-Champaign
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-08-15 至 2021-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1839010&HistoricalAwards=false
• 关键词: EAGER; Preserve; Destroy; Decisions; Simulation

The scientific research community has been increasingly developing ways to share and re-use research data, thereby allowing more discoveries to be made from previous research investments. Much of the focus on sharing and reusability has been on experimental and observational data. This project addresses the equally vexing challenge of how to make best use and re-use of the massive data produced in computational simulations. Important research questions guiding this project include: the degree to which simulation results can be replicated; the advantages of storing the simulation data itself for others to reuse as compared to providing the computational software so that others can re-run the simulations; and understanding which software testing practices can facilitate the replication/reuse of simulation data and the simulation software that produces those data. The principal investigators will address these questions by performing extensive replication and software code testing on a set of computational physics simulation datasets and software code that they had gathered through a previous study. The project will produce publicly available, fully reproducible computational physics works as examples for publishing results in a way that the data and code are effectively reusable.The principal investigators aim to improve understanding of, and increase, the reusability of the code and data associated with simulation-based research. This project specifically aims to better inform data destroy/preservation decisions in the simulation context, toward improving the reusability and interoperability of simulation data and code. The project will also consider important questions such as how software engineering testing practices relate to computational physics practices, and how changes in computational environments affect code execution and the regeneration of simulation data. Ultimately, the results of this work are intended to guide the research community on how to best produce and disseminate research code. It is anticipated that the results for computational physics can be extended to develop general guidelines for simulation data and code sharing for other communities, the appropriate code testing to do so, and best practices for development of associated cyberinfrastructure and tools. This project is supported by the National Science Foundation's Public Access Initiative which is managed by the NSF Office of Advanced Cyberinfrastructure on behalf of the Foundation.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

科学研究界一直在越来越多地开发共享和重复使用研究数据的方法，从而允许从以前的研究投资中获得更多发现。对共享和可重复性的重点是实验和观察数据。该项目解决了如何充分利用和重复计算模拟中产生的大量数据的同样烦人的挑战。引导该项目的重要研究问题包括：可以复制模拟结果的程度；与提供计算软件相比，存储仿真数据本身以使其他人重复使用的优点可以重新运行模拟；并了解哪种软件测试实践可以促进模拟数据的复制/重复使用以及产生这些数据的仿真软件。首席研究人员将通过对他们通过先前研究收集的一组计算物理模拟数据集和软件代码进行广泛的复制和软件代码测试来解决这些问题。该项目将产生公开可用的，完全可重现的计算物理学作为示例发布结果的示例，以便有效地重复使用数据和代码。 该项目专门旨在更好地告知数据在模拟上下文中的数据破坏/保存决策，以改善模拟数据和代码的可重复性和互操作性。该项目还将考虑重要的问题，例如软件工程测试实践如何与计算物理实践有关，以及计算环境的变化如何影响代码执行和模拟数据的再生。最终，这项工作的结果旨在指导研究社区如何最好地制作和传播研究法规。可以预期，可以扩展计算物理学的结果，以制定用于其他社区的仿真数据和代码共享，适当的代码测试以及开发相关网络基础结构和工具的最佳实践的一般准则。该项目得到了国家科学基金会的公共访问计划的支持，该计划由NSF高级网络基础架构办公室代表基金会管理。该奖项反映了NSF的法定任务，并被认为是通过基金会的知识分子优点和更广泛影响的审查标准来评估值得通过评估的。

项目成果

Domain-Specific Fixes for Flaky Tests with Wrong Assumptions on Underdetermined Specifications

• DOI: 10.1109/icse43902.2021.00018
• 发表时间: 2021-05
• 期刊: 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE)
• 作者: Peilun Zhang;Yanjie Jiang;Anjiang Wei;V. Stodden;D. Marinov;A. Shi

Understanding Reproducibility and Characteristics of Flaky Tests Through Test Reruns in Java Projects

• DOI: 10.1109/issre5003.2020.00045
• 发表时间: 2020-01-01
• 期刊: 2020 IEEE 31ST INTERNATIONAL SYMPOSIUM ON SOFTWARE RELIABILITY ENGINEERING (ISSRE 2020)
• 作者: Lam, Wing;Winter, Stefan;Marinov, Darko
• 通讯作者: Marinov, Darko

Building a Vision for Reproducibility in the Cyberinfrastructure Ecosystem: Leveraging Community Efforts

构建网络基础设施生态系统可重复性的愿景：利用社区的努力

• DOI: 10.14529/jsfi200106
• 发表时间: 2020
• 期刊: Supercomputing Frontiers and Innovations
• 作者: Chapp, D.;Stodden, V.;Taufer, M.
• 通讯作者: Taufer, M.

iFixFlakies: a framework for automatically fixing order-dependent flaky tests

• DOI: 10.1145/3338906.3338925
• 发表时间: 2019-08
• 期刊: Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering
• 作者: A. Shi;Wing Lam;Reed Oei;Tao Xie;D. Marinov

Empirically revisiting and enhancing IR-based test-case prioritization

• DOI: 10.1145/3395363.3397383
• 发表时间: 2020-07
• 期刊: Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis
• 作者: Qianyang Peng;A. Shi;Lingming Zhang