Collaborative Research: Frameworks: Convergence of Bayesian inverse methods and scientific machine learning in Earth system models through universal differentiable programming

协作研究：框架：通过通用可微编程将贝叶斯逆方法和科学机器学习在地球系统模型中融合

• 批准号: 2103942
• 负责人: Patrick Heimbach
• 金额: $ 127.9万
• 依托单位: University of Texas at Austin
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-08-01 至 2025-07-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2103942&HistoricalAwards=false
• 关键词: Collaborative; Research; Frameworks; Convergence; Bayesian

Understanding and quantifying parameter sensitivity of simulated systems, such as the numerical models of physical systems and mathematical renderings of neural networks, are essential in simulation-based science (SBS) and scientific machine learning (SciML). They are the key ingredients in Bayesian inference and neural network training. Seizing on the opportunity of emerging open-source Earth system model development in the Julia high-level programming language, this project is endowing these open-source models with automatic differentiation (AD) enabled derivative information, making these converging data science and simulation-based science tools available to a much broader research and data science community. Enabling a general-purpose AD framework which can handle both large-scale Earth system models as well as SciML algorithms, such as physics-informed neural networks or neural differential equations, will enable seamless integration of these approaches for hybrid Bayesian inversion and Bayesian machine learning. It merges big data science, in which available data enable model discovery with sparse data science, and the model structure is exploited in the selection of surrogate models representing data-informed subspaces and fulfilling conservation laws. The emerging Julia language engages a new generation of researchers and software engineers, channeling much needed talent into computational science approaches to climate modeling. Through dedicated community outreach programs (e.g., Hackathons, Minisymposia, Tutorials) the project team will be working toward increasing equity, diversity, and inclusion across the participating disciplines.The project is developing a framework for universal differentiable programming and open-source, general-purpose AD that unifies these algorithmic frameworks within Julia programming language. The general-purpose AD framework in Julia leverages the composability of Julia software packages and the differentiable programming approach that underlies many of the SciML and high-performance scientific computing packages. Compared to most current modeling systems targeted for HPC, Julia is ideally suited for heterogeneous parallel computing hardware (e.g., CUDA, ROCm, oneAPI, ARM, PowerPC, x86 64, TPUs). The project is bringing together expertise in AD targeted at Earth system data assimilation in high performance computing environments with SciML expertise. The project team is working with the Julia Computing organization and package developers to ensure sustainability of the developed frameworks. The project’s Earth system flagship applications consist of (i) an open-source, AD-enabled ocean general circulation model that is being developed separately as part of the Climate Modelling Alliance (CliMA), and (2) an open-source, AD-enabled ice flow model. Each of these application frameworks is being made available to the community for science application, in which derivative (gradient or Hessian) information represent key algorithmic enabling tools. These include SciML-based training of surrogate models (data-driven and/or model-informed), parameter and state estimation, data assimilation for model initialization, uncertainty quantification (Hessian-based and gradient-informed MCMC) and quantitative observing system design. Academic and industry partners are involved, who are using the frameworks for developing efficient power grids, personalized precision pharmacometrics, and improved EEG design.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

在基于模拟的科学（SBS）和科学机器学习（SCIML）中，了解和量化模拟系统的参数敏感性，例如物理系统的数值模型和神经网络的数学渲染。它们是贝叶斯推论和神经网络培训的关键归纳。该项目抓住了朱莉娅高级编程语言中新兴的开源地球系统模型开发的机会，该项目正在赋予这些开源模型具有自动差异化（AD）启用衍生派生信息，从而使这些收敛的数据科学和基于模拟的科学工具可用于更广泛的研究和数据科学社区。启用可以处理大规模接地系统模型以及SCIML算法的通用AD框架，例如物理知识的神经元网络或神经元微分方程，将无缝整合这些方法，以实现流质贝叶斯逆转录和Bayesian Inversion和Bayesian Machine学习。它合并了大数据科学，其中可用的数据使模型发现与稀疏数据科学，并在代表数据信息空间和实现保护法的替代模型的选择中探索了模型结构。新兴的朱莉娅语言与新一代的研究人员和软件工程师联系在一起，将急需的人才引入了计算科学的攀登模型方法。通过敬业的社区外展计划（例如，黑客马拉松，迷你典意，教程），项目团队将致力于增加参与学科的公平，多样性和包容性。该项目正在为这些Algorithmic segrancoms in julia segralmitss in julia segrallia semporments of templeliass开发一个通用的可区分编程和开放式编程和开放式编程的框架。朱莉娅（Julia）中的通用广告框架利用了朱莉娅软件包的合成性和可区分的编程方法，这些方法是许多SCIML和高性能科学计算套件的基础。与针对HPC的大多数当前建模系统相比，Julia非常适合异质并行计算硬件（例如CUDA，ROCM，ONEAPI，ONEAPI，ONEAPI，ARM，POWERPC，POWERPC，X86 64，TPU）。该项目将针对地球系统数据同化的广告中的专业知识汇总在具有SCIML专业知识的高性能计算环境中。项目团队正在与朱莉娅计算机组织和包装开发人员合作，以确保开发框架的可持续性。该项目的地球系统旗舰应用程序由（i）开源，启用广告的海洋循环模型，该模型正在作为气候建模联盟（CLIMA）的一部分分别开发，以及（2）开源的，具有ad的冰流模型。这些应用程序框架中的每一个都可以提供给社区的科学应用程序，其中衍生（梯度或黑森）信息代表关键算法启用工具。其中包括基于SCIML的替代模型培训（数据驱动和/或模型信息），参数和状态估计，模型初始化的数据同化，不确定性定量（基于Hessian的和基于梯度的MCMC）以及定量观察者系统设计。涉及学术和行业合作伙伴，他们正在使用框架来开发有效的电网，个性化的精确药物计量学和改进的EEG设计。该奖项反映了NSF的法定任务，并被认为是通过基金会的知识分子优点和更广泛的影响审查标准来评估来获得的支持。

项目成果

Reverse-Mode Automatic Differentiation and Optimization of GPU Kernels via Enzyme

• DOI: 10.1145/3458817.3476165
• 发表时间: 2021-11
• 期刊: SC21: International Conference for High Performance Computing, Networking, Storage and Analysis
• 作者: William S. Moses;Valentin Churavy;Ludger Paehler;J. Hückelheim;S. Narayanan;Michel Schanen;J. Doerfert

The Computational Science of Klaus Hasselmann

• DOI: 10.1109/mcse.2022.3195105
• 发表时间: 2022-07
• 期刊: Computing in Science & Engineering
• 影响因子: 2.1
• 作者: P. Heimbach

Road Map for the Next Decade of Earth System Reanalysis in the United States

美国未来十年地球系统再分析路线图

• DOI: 10.1175/bams-d-23-0011.1
• 发表时间: 2023
• 期刊: Bulletin of the American Meteorological Society
• 影响因子: 8
• 作者: Frolov, Sergey;Rousseaux, Cécile S.;Auligne, Tom;Dee, Dick;Gelaro, Ron;Heimbach, Patrick;Simpson, Isla;Slivinski, Laura
• 通讯作者: Slivinski, Laura

SICOPOLIS-AD v2: tangent linear and adjoint modelingframework for ice sheet modeling enabled by automatic differentiationtool Tapenade

SICOPOLIS-AD v2：由自动微分工具 Tapenade 支持的冰盖建模的切线线性和伴随建模框架

• DOI: 10.21105/joss.04679
• 发表时间: 2023
• 期刊: Journal of Open Source Software
• 作者: Gaikwad, Shreyas Sunil;Hascoet, Laurent;Narayanan, Sri Hari;Curry-Logan, Liz;Greve, Ralf;Heimbach, Patrick
• 通讯作者: Heimbach, Patrick