EAGER: Cyberinfrastructure Reproducibility Project: Computational Science and Engineering

EAGER：网络基础设施再现性项目：计算科学与工程

基本信息

批准号：
1747669
负责人：
Lorena Barba
金额：
$ 29.98万
依托单位：
George Washington University
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2017
资助国家：
美国
起止时间：
2017-09-01 至 2021-08-31
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1747669&HistoricalAwards=false
关键词：
EAGER Cyberinfrastructure Reproducibility Project Computational

项目摘要

The progress of science today relies heavily on the use of computers, from lab workstations to national supercomputers. This project confronts the issue of ensuring that scientific knowledge stemming from computational research complies to high standards of reproducibility and rigor. The NSF CISE Directorate "Dear Colleague Letter: Encouraging Reproducibility in Computing and Communications Research" (NSF 17-022, 2016) encourages researchers to embrace completeness and transparency in developing rigorous protocols. Reproducibility concerns often focus on transparency via open data, code, and other research objects. But providing the data and code to run all analyses again is, however, a minimum standard. In the context of high-performance computing where research uses multi-million-dollar facilities the understanding of where lie the sources of non-reproducibility is still lacking. This project uses methodical replication of previous studies in computational fluid dynamics as the model for reaching that understanding and for developing the guiding principles of study design that can guarantee, as much as possible, reproducible findings. The results of this project will serve NSF's mission to promote the progress of science; to advance the national health, prosperity and welfare; to secure the national defense by bringing new and necessary understanding about how to achieve rigorous, reproducible computational research. This project conducts methodical replication of published studies in computational fluid dynamics, as a model for the broad field of computational science and engineering. Its aims are to 1) identify and characterize sources of non-reproducibility, and 2) develop guidelines on Design for Reproducibility, i.e., study design guaranteeing reproducible findings. The project addresses the role of scientific software libraries (like linear algebra solvers), the influence of hardware architectures , the role of new technologies for reproducible research (e.g., containers, cloud services). The project also provides guidelines for making assessments about the success of replication studies in the context of high-performance computing, including when it is reasonable or not to expect bit-by-bit numerical reproducibility. The research tackles three phases of computational research: Research methods; Research communication; Research assessment. The project also develops openly licensed training materials on reproducible computational research, including a graduate seminar course and a Responsible Conduct of Research (RCR) module.

当今科学的进步在很大程度上依赖于计算机的使用，从实验室工作站到国家超级计算机。该项目面临的问题是确保源自计算研究的科学知识符合高标准的可重复性和严谨性。 NSF CISE 理事会“亲爱的同事的信：鼓励计算和通信研究的可重复性”（NSF 17-022，2016）鼓励研究人员在开发严格的协议时拥抱完整性和透明度。可重复性问题通常集中在开放数据、代码和其他研究对象的透明度上。但提供数据和代码以再次运行所有分析是最低标准。在高性能计算的背景下，研究使用数百万美元的设施，但仍然缺乏对不可再现性来源的理解。该项目使用计算流体动力学中先前研究的系统复制作为模型，以达成这种理解并制定研究设计的指导原则，以尽可能保证可重复的结果。该项目的成果将服务于NSF促进科学进步的使命；促进国民健康、繁荣和福利；通过对如何实现严格、可重复的计算研究带来新的和必要的理解来确保国防安全。该项目对已发表的计算流体动力学研究进行了系统的复制，作为计算科学和工程广泛领域的模型。其目标是 1) 识别和描述不可重复性的来源，2) 制定可重复性设计指南，即保证可重复结果的研究设计。该项目解决了科学软件库（如线性代数求解器）的作用、硬件架构的影响、可重复研究新技术（例如容器、云服务）的作用。该项目还提供了在高性能计算背景下评估复制研究成功与否的指南，包括何时合理或不期望逐位数值再现性。该研究涉及计算研究的三个阶段：研究方法；研究交流；研究评估。该项目还开发了有关可重复计算研究的公开许可培训材料，包括研究生研讨会课程和负责任的研究行为（RCR）模块。