OAC Core: The Best of Both Worlds: Deep Neural Operators as Preconditioners for Physics-Based Forward and Inverse Problems

OAC 核心：两全其美：深度神经算子作为基于物理的正向和逆向问题的预处理器

• 批准号: 2313033
• 负责人: Omar Ghattas
• 金额: $ 60万
• 依托单位: University of Texas at Austin
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-09-01 至 2026-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2313033&HistoricalAwards=false
• 关键词: OAC; Core; Best; Both; Worlds

Many physical systems across all areas of science, engineering, medicine, and defense are modeled with high accuracy by partial differential equations (PDEs) and solved on advanced computing systems. Often the ultimate goal is to repeatedly solve the PDEs to explore parameter uncertainties. Settings in which this arises are inverse problems (inferring uncertain parameters of a model from data), optimal experimental design (determining the optimal data acquisition to learn the most about the model), optimal design (finding the optimal configuration of a system to maximize performance), and optimal control (determining the optimal operation of a system to achieve a desired behavior). These problems are often characterized by high dimensional uncertain parameter spaces, since the parameters typically represent initial conditions, boundary conditions, material properties, or source terms and vary in space and/or time. As a result, the PDEs often have to be solved thousands or even millions of times to adequately represent uncertainties in the parameters. When the systems that are modeled involve coupled multiple physics or behavior occurring on multiple space and time scales, repeated solution of the PDE models becomes prohibitive, even on the latest supercomputers. The development of deep neural networks in recent years shows promise in overcoming the intractability of repeated solution of the PDE models, by learning the relationships between the input parameters and the outputs of interest (e.g., temperature, velocity, pressure, stress, electric field, magnetic field, chemical species). Once trained on PDE solution data, the networks can evaluate the outputs for any given inputs in milliseconds, compared to hours or days to solve the PDE models themselves. However, despite much progress in the development of these so-called neural network surrogates, they typically deliver just 1-2 digits of accuracy, which is not sufficient to replace the PDE solver. Instead, this project is developing hybrids of neural network surrogates and PDE models that combine the best properties of each: the accuracy of the PDEs with the speed of the neural networks. The impact is that many problems in technology, health, the environment, and society that were not amenable to complex model-based inference and decision making will now become tractable. The algorithms developed in this project are being released as open-source software so that a broad community of researchers and practitioners can apply them to a spectrum of scientific and engineering problems. In addition, the surrogate methods developed in this project are being incorporated into a popular graduate course on inverse problems taught at University of Texas, Austin.Neural network approximations of high fidelity PDE solutions, i.e., neural operators, have gained popularity in recent years due to their ease of implementation, adaptability to varied settings, and seeming ability to mitigate the curse of dimensionality. Significant recent research has attempted to establish "universal approximation" properties of these surrogates for various classes of maps. While theory suggests that neural operators can in principle achieve arbitrary accuracy, realizing this in practice remains a significant challenge. The reasons for this include the enormous costs of generating sufficient training data, and confounding relations between statistical sampling errors, approximation errors, and nonconvexity of the training problem. Often neural operators can achieve just 1-2 digits of accuracy relative to high fidelity PDE solvers, with little hope of further reducing this accuracy. On the other hand, high fidelity PDE models (particularly conservation and balance laws) are often known with very high confidence and high precision is necessary due to sensitivity of PDE solutions to small perturbations in the inputs. The modest accuracies of neural operators are often insufficient for the demands of inference, control, and decision making for critical systems. This project is developing hybrids of neural operators and high fidelity PDE models to realize the best features of each, by retaining accuracy via the PDE residual and speed via use of the neural operator as a preconditioner. The project targets linear and nonlinear parametric neural preconditioners for PDEs, and neural preconditioners for Metropolized Langevin methods to accelerate the solution of Bayesian inverse problems. A further advantage of using neural operators as preconditioners is that they map well onto GPU architectures.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

科学、工程、医学和国防各个领域的许多物理系统都通过偏微分方程 (PDE) 进行高精度建模，并在先进的计算系统上求解。通常，最终目标是重复求解偏微分方程以探索参数不确定性。出现这种情况的设置是逆问题（从数据推断模型的不确定参数）、最优实验设计（确定最佳数据采集以最大限度地了解模型）、最优设计（找到系统的最佳配置以最大化性能） ）和最优控制（确定系统的最优运行以实现所需的行为）。这些问题通常以高维不确定参数空间为特征，因为参数通常表示初始条件、边界条件、材料属性或源项，并且在空间和/或时间上变化。因此，偏微分方程通常需要求解数千甚至数百万次才能充分表示参数中的不确定性。当建模的系统涉及多个空间和时间尺度上发生的耦合多个物理或行为时，即使在最新的超级计算机上，偏微分方程模型的重复求解也变得令人望而却步。近年来深度神经网络的发展表明，通过学习输入参数和感兴趣的输出（例如温度、速度、压力、应力、电场、磁场、化学物质）。一旦接受了偏微分方程求解数据的训练，网络就可以在几毫秒内评估任何给定输入的输出，而求解偏微分方程模型本身需要花费数小时或数天的时间。然而，尽管这些所谓的神经网络代理的开发取得了很大进展，但它们通常只能提供 1-2 位数的精度，这不足以取代 PDE 求解器。相反，该项目正在开发神经网络代理和偏微分方程模型的混合体，结合了各自的最佳特性：偏微分方程的准确性和神经网络的速度。其影响是，技术、健康、环境和社会中的许多原本无法通过复杂的基于模型的推理和决策来解决的问题现在将变得易于处理。该项目开发的算法将作为开源软件发布，以便广泛的研究人员和从业者社区可以将它们应用于一系列科学和工程问题。此外，该项目中开发的替代方法正在被纳入德克萨斯大学奥斯汀分校教授的一门流行的反问题研究生课程中。高保真 PDE 解决方案的神经网络近似（即神经算子）近年来越来越受欢迎，因为它们易于实施、对不同环境的适应性以及减轻维度灾难的能力。最近的重要研究试图为各类地图建立这些替代物的“通用近似”属性。虽然理论表明神经算子原则上可以实现任意精度，但在实践中实现这一点仍然是一个重大挑战。其原因包括生成足够训练数据的巨大成本，以及统计采样误差、近似误差和训练问题的非凸性之间的混杂关系。相对于高保真 PDE 求解器，神经算子通常只能实现 1-2 位数的精度，并且几乎没有希望进一步降低这种精度。另一方面，高保真 PDE 模型（特别是守恒定律和平衡律）通常具有非常高的置信度，并且由于 PDE 解对输入中的小扰动的敏感性，高精度是必要的。神经算子的适度精度通常不足以满足关键系统的推理、控制和决策的要求。该项目正在开发神经算子和高保真 PDE 模型的混合体，通过 PDE 残差保持精度，并通过使用神经算子作为预处理器保持速度，从而实现各自的最佳功能。该项目的目标是偏微分方程的线性和非线性参数神经预处理器以及都市朗之万方法的神经预处理器，以加速贝叶斯逆问题的求解。使用神经算子作为预处理器的另一个优点是它们可以很好地映射到 GPU 架构上。该奖项反映了 NSF 的法定使命，并且通过使用基金会的智力优点和更广泛的影响审查标准进行评估，被认为值得支持。