OAC Core: The Best of Both Worlds: Deep Neural Operators as Preconditioners for Physics-Based Forward and Inverse Problems

OAC 核心：两全其美：深度神经算子作为基于物理的正向和逆向问题的预处理器

• 批准号: 2313033
• 负责人: Omar Ghattas
• 金额: $ 60万
• 依托单位: University of Texas at Austin
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-09-01 至 2026-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2313033&HistoricalAwards=false
• 关键词: OAC; Core; Best; Both; Worlds

Many physical systems across all areas of science, engineering, medicine, and defense are modeled with high accuracy by partial differential equations (PDEs) and solved on advanced computing systems. Often the ultimate goal is to repeatedly solve the PDEs to explore parameter uncertainties. Settings in which this arises are inverse problems (inferring uncertain parameters of a model from data), optimal experimental design (determining the optimal data acquisition to learn the most about the model), optimal design (finding the optimal configuration of a system to maximize performance), and optimal control (determining the optimal operation of a system to achieve a desired behavior). These problems are often characterized by high dimensional uncertain parameter spaces, since the parameters typically represent initial conditions, boundary conditions, material properties, or source terms and vary in space and/or time. As a result, the PDEs often have to be solved thousands or even millions of times to adequately represent uncertainties in the parameters. When the systems that are modeled involve coupled multiple physics or behavior occurring on multiple space and time scales, repeated solution of the PDE models becomes prohibitive, even on the latest supercomputers. The development of deep neural networks in recent years shows promise in overcoming the intractability of repeated solution of the PDE models, by learning the relationships between the input parameters and the outputs of interest (e.g., temperature, velocity, pressure, stress, electric field, magnetic field, chemical species). Once trained on PDE solution data, the networks can evaluate the outputs for any given inputs in milliseconds, compared to hours or days to solve the PDE models themselves. However, despite much progress in the development of these so-called neural network surrogates, they typically deliver just 1-2 digits of accuracy, which is not sufficient to replace the PDE solver. Instead, this project is developing hybrids of neural network surrogates and PDE models that combine the best properties of each: the accuracy of the PDEs with the speed of the neural networks. The impact is that many problems in technology, health, the environment, and society that were not amenable to complex model-based inference and decision making will now become tractable. The algorithms developed in this project are being released as open-source software so that a broad community of researchers and practitioners can apply them to a spectrum of scientific and engineering problems. In addition, the surrogate methods developed in this project are being incorporated into a popular graduate course on inverse problems taught at University of Texas, Austin.Neural network approximations of high fidelity PDE solutions, i.e., neural operators, have gained popularity in recent years due to their ease of implementation, adaptability to varied settings, and seeming ability to mitigate the curse of dimensionality. Significant recent research has attempted to establish "universal approximation" properties of these surrogates for various classes of maps. While theory suggests that neural operators can in principle achieve arbitrary accuracy, realizing this in practice remains a significant challenge. The reasons for this include the enormous costs of generating sufficient training data, and confounding relations between statistical sampling errors, approximation errors, and nonconvexity of the training problem. Often neural operators can achieve just 1-2 digits of accuracy relative to high fidelity PDE solvers, with little hope of further reducing this accuracy. On the other hand, high fidelity PDE models (particularly conservation and balance laws) are often known with very high confidence and high precision is necessary due to sensitivity of PDE solutions to small perturbations in the inputs. The modest accuracies of neural operators are often insufficient for the demands of inference, control, and decision making for critical systems. This project is developing hybrids of neural operators and high fidelity PDE models to realize the best features of each, by retaining accuracy via the PDE residual and speed via use of the neural operator as a preconditioner. The project targets linear and nonlinear parametric neural preconditioners for PDEs, and neural preconditioners for Metropolized Langevin methods to accelerate the solution of Bayesian inverse problems. A further advantage of using neural operators as preconditioners is that they map well onto GPU architectures.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

科学，工程，医学和防御各个领域的许多物理系统都是通过部分微分方程（PDE）高精度建模的，并在高级计算系统上解决。通常，最终目标是反复解决PDE，以探索参数不确定性。发生这种情况的设置是反问题（从数据中推断模型的不确定参数），最佳实验设计（确定最佳数据采集以最大程度地了解该模型），最佳设计（找到系统的最佳配置以最大程度地提高性能）和最佳控制（确定系统的最佳操作以实现达到期望的行为）。这些问题通常以高维不确定的参数空间为特征，因为这些参数通常代表初始条件，边界条件，材料属性或源术语，并且空间和/或时间有所不同。结果，PDE通常必须解决数千甚至数百万次以充分代表参数中的不确定性。当建模的系统涉及在多个空间和时间尺度上发生的多个物理或行为耦合时，即使在最新的超级计算机上，PDE模型的重复解决方案也变得越来越高。近年来，深度神经网络的发展通过学习输入参数与感兴趣的输出之间的关系（例如温度，速度，压力，压力，电场，电场，磁场，化学物种），通过学习输入参数与感兴趣的输出之间的关系来克服重复解决方案的棘手性。一旦对PDE解决方案数据进行了培训，网络可以评估毫秒中任何给定输入的输出，而小时或几天可以解决PDE模​​型本身。但是，尽管这些所谓的神经网络代理的发展取得了很大进展，但它们通常仅提供1-2位准确性数字，这不足以替代PDE求解器。取而代之的是，该项目正在开发结合了每种最佳特性的神经网络替代和PDE模型的混合体：PDE的准确性与神经网络的速度。影响的是，技术，健康，环境和社会中不适合基于复杂模型的推理和决策的许多问题现在将变得易于处理。该项目中开发的算法是作为开源软件发布的，以便广泛的研究人员和从业人员可以将其应用于科学和工程问题。此外，该项目中开发的替代方法已被纳入德克萨斯大学奥斯汀大学教授的流行研究生课程中。高保真性PDE解决方案的神经网络近似，即神经操作员，近年来由于实施的能力，可适应各种设置的能力，并且看起来像是杂乱无章的杂物，近年来近年来越来越受欢迎。最近的大量研究试图建立这些替代物在各种地图上的“通用近似”特性。尽管理论认为神经操作员原则上可以实现任意准确性，但实践中意识到这仍然是一个重大挑战。这样做的原因包括产生足够的培训数据的巨大成本，以及统计抽样错误，近似错误和培训问题的非概念之间的混淆。通常，神经操作员只能获得相对于高保真求解器的1-2位准确性数字，几乎没有希望进一步降低这种准确性。另一方面，高保真PDE模型（尤其是保护和平衡定律）通常以很高的置信度知道，因此由于PDE溶液对输入中的小扰动的敏感性，因此需要高精度。神经操作员的适度精度通常不足以满足对关键系统的推理，控制和决策的需求。该项目正在开发神经操作员和高保真PDE模型的混合体，以通过使用神经操作员用作预处理器来通过PDE残差和速度来实现每个特征的最佳功能。该项目针对PDE的线性和非线性参数神经预调节器，以及用于大都市Langevin方法的神经预处理，以加速贝叶斯逆问题的解决方案。将神经操作员用作预调查人员的另一个优点是它们很好地映射到了GPU架构上。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的知识分子优点和更广泛的影响评估标准通过评估来支持的。