Collaborative Research: Frameworks: Convergence of Bayesian inverse methods and scientific machine learning in Earth system models through universal differentiable programming

协作研究：框架：通过通用可微编程将贝叶斯逆方法和科学机器学习在地球系统模型中融合

• 批准号: 2103942
• 负责人: Patrick Heimbach
• 金额: $ 127.9万
• 依托单位: University of Texas at Austin
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-08-01 至 2025-07-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2103942&HistoricalAwards=false
• 关键词: Collaborative; Research; Frameworks; Convergence; Bayesian

Understanding and quantifying parameter sensitivity of simulated systems, such as the numerical models of physical systems and mathematical renderings of neural networks, are essential in simulation-based science (SBS) and scientific machine learning (SciML). They are the key ingredients in Bayesian inference and neural network training. Seizing on the opportunity of emerging open-source Earth system model development in the Julia high-level programming language, this project is endowing these open-source models with automatic differentiation (AD) enabled derivative information, making these converging data science and simulation-based science tools available to a much broader research and data science community. Enabling a general-purpose AD framework which can handle both large-scale Earth system models as well as SciML algorithms, such as physics-informed neural networks or neural differential equations, will enable seamless integration of these approaches for hybrid Bayesian inversion and Bayesian machine learning. It merges big data science, in which available data enable model discovery with sparse data science, and the model structure is exploited in the selection of surrogate models representing data-informed subspaces and fulfilling conservation laws. The emerging Julia language engages a new generation of researchers and software engineers, channeling much needed talent into computational science approaches to climate modeling. Through dedicated community outreach programs (e.g., Hackathons, Minisymposia, Tutorials) the project team will be working toward increasing equity, diversity, and inclusion across the participating disciplines.The project is developing a framework for universal differentiable programming and open-source, general-purpose AD that unifies these algorithmic frameworks within Julia programming language. The general-purpose AD framework in Julia leverages the composability of Julia software packages and the differentiable programming approach that underlies many of the SciML and high-performance scientific computing packages. Compared to most current modeling systems targeted for HPC, Julia is ideally suited for heterogeneous parallel computing hardware (e.g., CUDA, ROCm, oneAPI, ARM, PowerPC, x86 64, TPUs). The project is bringing together expertise in AD targeted at Earth system data assimilation in high performance computing environments with SciML expertise. The project team is working with the Julia Computing organization and package developers to ensure sustainability of the developed frameworks. The project’s Earth system flagship applications consist of (i) an open-source, AD-enabled ocean general circulation model that is being developed separately as part of the Climate Modelling Alliance (CliMA), and (2) an open-source, AD-enabled ice flow model. Each of these application frameworks is being made available to the community for science application, in which derivative (gradient or Hessian) information represent key algorithmic enabling tools. These include SciML-based training of surrogate models (data-driven and/or model-informed), parameter and state estimation, data assimilation for model initialization, uncertainty quantification (Hessian-based and gradient-informed MCMC) and quantitative observing system design. Academic and industry partners are involved, who are using the frameworks for developing efficient power grids, personalized precision pharmacometrics, and improved EEG design.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

理解和量化模拟系统的参数敏感性，例如物理系统的数值模型和神经网络的数学渲染，对于基于模拟的科学 (SBS) 和科学机器学习 (SciML) 至关重要，它们是贝叶斯推理的关键要素。该项目抓住了用 Julia 高级自动编程语言开发新兴开源地球系统模型的机会，以支持微分（AD）的衍生信息结束这些开源模型，使这些数据趋于一致。科学和为更广泛的研究和数据科学界提供基于模拟的科学工具，实现通用 AD 框架，该框架可以处理大规模地球系统模型以及 SciML 算法，例如基于物理的神经网络或神经微分方程。 ，将实现混合贝叶斯反演和贝叶斯机器学习的这些方法的无缝集成，它融合了大数据科学，其中可用数据使模型发现与稀疏数据科学相结合，并且在选择代表的代理模型时利用模型结构。新兴的 Julia 语言吸引了新一代研究人员和软件工程师，通过专门的社区推广计划（例如黑客马拉松、小型研讨会、教程）将急需的人才引入气候建模的计算科学方法。项目团队将致力于增加参与学科的公平性、多样性和包容性。该项目正在开发一个通用可微分编程和开源通用 AD 框架，将这些算法框架统一到 Julia 编程中Julia 中的通用 AD 框架利用了 Julia 软件包的可组合性以及作为许多 SciML 和高性能科学计算包基础的可微分编程方法，与大多数当前针对 HPC 的建模系统相比，Julia 非常适合。该项目汇集了针对地球系统数据同化的 AD 专业知识。该项目团队正在与 Julia 计算组织和软件包开发人员合作，以确保该项目的地球系统旗舰应用程序包括 (i) 一个开源的、支持 AD 的海洋。作为气候建模联盟 (CliMA) 的一部分正在单独开发的大气环流模型，以及 (2) 开源的、支持 AD 的冰流模型中的每一个都可供社区用于科学应用。 ，其中导数（梯度或 Hessian）信息代表关键的算法支持工具，其中包括基于 SciML 的代理模型训练（数据驱动和/或模型通知）、参数和状态估计、模型初始化的数据同化、不确定性量化（基于 Hessian）。和梯度信息 MCMC）以及定量观察系统设计参与其中，他们正在使用开发高效电网、个性化精确药理学和改进脑电图的框架。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Road Map for the Next Decade of Earth System Reanalysis in the United States

美国未来十年地球系统再分析路线图

• DOI: 10.1175/bams-d-23-0011.1
• 发表时间: 2023-03
• 期刊: Bulletin of the American Meteorological Society
• 影响因子: 8
• 作者: Frolov, Sergey;Rousseaux, Cécile S.;Auligne, Tom;Dee, Dick;Gelaro, Ron;Heimbach, Patrick;Simpson, Isla;Slivinski, Laura
• 通讯作者: Slivinski, Laura

The Computational Science of Klaus Hasselmann

克劳斯·哈塞尔曼的计算科学

• DOI: 10.1109/mcse.2022.3195105
• 发表时间: 2022-07-01
• 期刊: Computing in Science & Engineering
• 影响因子: 2.1
• 作者: P. Heimbach

SICOPOLIS-AD v2: tangent linear and adjoint modelingframework for ice sheet modeling enabled by automatic differentiationtool Tapenade

SICOPOLIS-AD v2：由自动微分工具 Tapenade 支持的冰盖建模的切线线性和伴随建模框架

• DOI: 10.21105/joss.04679
• 发表时间: 2023-03
• 期刊: Journal of Open Source Software
• 作者: Gaikwad, Shreyas Sunil;Hascoet, Laurent;Narayanan, Sri Hari;Curry;Greve, Ralf;Heimbach, Patrick
• 通讯作者: Heimbach, Patrick

Reverse-mode automatic differentiation and optimization of GPU kernels via enzyme

通过酶逆向模式自动微分和优化 GPU 内核

• DOI: 10.1145/3458817.3476165
• 发表时间: 2021-11
• 期刊: Storage and Analysis
• 作者: Moses, William S.;Churavy, Valentin;Paehler, Ludger;Hückelheim, Jan;Narayanan, Sri Hari;Schanen, Michel;Doerfert, Johannes
• 通讯作者: Doerfert, Johannes