Collaborative Research: Theory and Applications of Structure-Conforming Deep Operator Learning

合作研究：结构符合深度算子学习的理论与应用

• 批准号: 2309778
• 负责人: Shuhao Cao
• 金额: $ 15.6万
• 依托单位: University of Missouri-Kansas City
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-06-01 至 2026-05-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2309778&HistoricalAwards=false
• 关键词: Collaborative; Research; Theory; Applications; Structure

The first-principle-based approach has achieved considerable success in numerous engineering and scientific disciplines, including fluid and solid mechanics, electromagnetism, and more. Among its most significant applications are partial differential equations (PDEs) which, in conjunction with their analysis and numerical algorithms, represent some of the most powerful tools humanity has ever developed for understanding the material world. However, increasingly complex mathematical models arising from physics, biology, and chemistry challenge the efficacy of first-principle-based approaches for solving practical problems, such as those in fluid turbulence, molecular dynamics, and large-scale inverse problems. A major obstacle for numerical algorithms is the so-called curse of dimensionality. Fueled by advances in Graphics Processing Unit and Tensor Processing Unit general-purpose computing, deep neural networks (DNNs) and deep learning approaches excel in combating the curse of dimensionality and demonstrate immense potential for solving complex problems in science and engineering. This project aims to investigate how mathematical structures within a problem can inform the design and analysis of innovative DNNs, particularly in the context of inverse problems where unknown parameters are inferred from measurements, such as electrical impedance tomography. Additionally, the programming component in this project will focus on training the next generation of computational mathematicians.The Operator Learning (OpL) framework in deep learning provides a unique perspective for tackling challenging and potentially ill-posed PDE-based problems. This project will explore the potential of OpL to mitigate the ill-posedness of many inverse problems, as its powerful approximation capability combined with offline training and online prediction properties lead to high-quality, rapid reconstructions. The project seeks to bridge OpL and classical methodologies by integrating mathematical structures from classical problem-solving approaches into DNN architectures. In particular, the project will shed light on the mathematical properties of the attention mechanism, the backbone of state-of-the-art DNN Transformers, such as those in GPT and AlphaFold 2. Furthermore, the project will examine the flexibility of attention neural architectures, enabling the fusion of attention mechanisms with important methodologies in applied mathematics, such as Galerkin projection or Fredholm integral equations, in accordance with the a priori mathematical structure of a problem. This project will also delve into the mathematical foundations of attention through the lens of spectral theory in Hilbert spaces, seeking to understand how the emblematic query-key-value architecture contributes to the rich representational power and diverse approximation capabilities of Transformers.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

基于第一原则的方法在众多工程和科学学科中取得了巨大的成功，包括流体和固体力学，电磁作用等。其最重要的应用之一是部分微分方程（PDE），与它们的分析和数值算法结合使用，它代表了人类为理解物质世界开发的一些最强大的工具。但是，源于物理，生物学和化学引起的日益复杂的数学模型挑战了基于第一原则的方法解决实践问题的功效，例如流体湍流，分子动力学和大规模反向问题。数值算法的主要障碍是所谓的维度诅咒。 图形处理单元和张量处理单元通用计算，深神经网络（DNN）和深度学习方法的进步助长了努力，在抗击维度的诅咒方面表现出色，并证明了解决科学和工程中复杂问题的巨大潜力。该项目旨在调查问题中的数学结构如何为创新DNN的设计和分析提供信息，尤其是在从测量值（例如电阻抗断层扫描）中推断出未知参数的背景下。此外，该项目中的编程组件将重点介绍培训下一代计算数学家。深度学习中的操作员学习（OPL）框架为解决具有挑战性且潜在的基于PDE的问题提供了独特的观点。该项目将探讨OPL减轻许多反问题的不良性的潜力，因为其强大的近似能力与离线培训和在线预测属性相结合，导致了高质量的快速重建。该项目试图通过将经典问题解决方法的数学结构整合到DNN体系结构中来弥合OPL和经典方法论。 In particular, the project will shed light on the mathematical properties of the attention mechanism, the backbone of state-of-the-art DNN Transformers, such as those in GPT and AlphaFold 2. Furthermore, the project will examine the flexibility of attention neural architectures, enabling the fusion of attention mechanisms with important methodologies in applied mathematics, such as Galerkin projection or Fredholm integral equations, in accordance with the问题的先验数学结构。该项目还将通过希尔伯特（Hilbert）空间中的光谱理论镜头来深入研究注意力基础，以了解象征性的查询 - 钥匙价值架构如何为变形金刚的丰富代表性和多样化的近似能力做出贡献。该奖项奖励NSF的法定任务，并反映了通过评估范围的范围的范围的范围，该奖项被认为是众所周知的范围。