Collaborative Research: Algorithms, Theory, and Validation of Deep Graph Learning with Limited Supervision: A Continuous Perspective

协作研究：有限监督下的深度图学习的算法、理论和验证：连续的视角

• 批准号: 2208361
• 负责人: Bao Wang
• 金额: $ 24万
• 依托单位: University of Utah
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-09-01 至 2025-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2208361&HistoricalAwards=false
• 关键词: Collaborative; Research; Algorithms; Theory; Validation

Graph-structured data is ubiquitous in scientific and artificial intelligence applications, for instance, particle physics, computational chemistry, drug discovery, neural science, recommender systems, robotics, social networks, and knowledge graphs. Graph neural networks (GNNs) have achieved tremendous success in a broad class of graph learning tasks, including graph node classification, graph edge prediction, and graph generation. Nevertheless, there are several bottlenecks of GNNs: 1) In contrast to many deep networks such as convolutional neural networks, it has been noticed that increasing the depth of GNNs results in a severe accuracy degradation, which has been interpreted as over-smoothing in the machine learning community. 2) The performance of GNNs relies heavily on a sufficient number of labeled graph nodes; the prediction of GNNs will become significantly less reliable when less labeled data is available. This research aims to address these challenges by developing new mathematical understanding of GNNs and theoretically-principled algorithms for graph deep learning with less training data. The project will train graduate students and postdoctoral associates through involvement in the research. The project will also integrate the research into teaching to advance data science education.This project aims to develop next-generation continuous-depth GNNs leveraging computational mathematics tools and insights and to advance data-driven scientific simulation using the new GNNs. This project has three interconnected thrusts that revolve around pushing the envelope of theory and practice in graph deep learning with limited supervision using PDE and harmonic analysis tools: 1) developing a new generation of diffusion-based GNNs that are certifiable to learning with deep architectures and less training data; 2) developing a new efficient attention-based approach for learning graph structures from the underlying data accompanied by uncertainty quantification; and 3) application validation in learning-assisted scientific simulation and multi-modal learning and software development.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

图形结构的数据在科学和人工智能应用中无处不在，例如粒子物理，计算化学，药物发现，神经科学，推荐系统，机器人技术，社交网络和知识图。图形神经网络（GNN）在广泛的图形学习任务中取得了巨大的成功，包括图节点分类，图形边缘预测和图形生成。然而，有几种GNN的瓶颈：1）与许多深层网络（例如卷积神经网络）相反，已经注意到，增加GNNS的深度会导致严重的准确性降解，这已被解释为机器学习社区中的过度易光度。 2）GNN的性能在很大程度上取决于足够数量的标记图节点；当可用的标记数据较少时，GNN的预测将变得明显降低。这项研究旨在通过对GNN和理论原理算法进行新的数学理解来解决这些挑战，以使用较少的培训数据来绘制深度学习。该项目将通过参与研究来培训研究生和博士后同事。该项目还将将研究整合到进步数据科学教育中。该项目旨在开发下一代连续深度的GNN，利用计算数学工具和见解，并使用新的GNN来推进数据驱动的科学模拟。该项目具有三个相互连接的推力，这些推力围绕着使用PDE和谐波分析工具有限的监督来推动理论和实践的信封：1）开发新一代的基于扩散的GNN，这些基于扩散的GNN可以通过深度建筑和较少的培训数据进行认证，并且可以进行学习； 2）从基础数据中开发一种新的基于注意力的方法来学习图形结构，并伴随着不确定性量化； 3）在学习辅助科学模拟和多模式学习和软件开发中的应用验证。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的知识分子优点和更广泛的影响评估标准通过评估来获得支持的。

项目成果

A deterministic gradient-based approach to avoid saddle points

一种避免鞍点的基于确定性梯度的方法

• DOI: 10.1017/s0956792522000316
• 发表时间: 2022
• 期刊: European Journal of Applied Mathematics
• 影响因子: 1.9
• 作者: Kreusser, L. M.;Osher, S. J.;Wang, B.
• 通讯作者: Wang, B.

Efficient and Reliable Overlay Networks for Decentralized Federated Learning

• DOI: 10.1137/21m1465081
• 发表时间: 2021-12
• 期刊: SIAM J. Appl. Math.
• 作者: Yifan Hua;Kevin Miller;A. Bertozzi;Chen Qian;Bao Wang

Accelerated Sparse Recovery via Gradient Descent with Nonlinear Conjugate Gradient Momentum

• DOI: 10.1007/s10915-023-02148-y
• 发表时间: 2022-08
• 期刊: Journal of Scientific Computing
• 影响因子: 2.5
• 作者: Mengqi Hu;Y. Lou;Bao Wang;Ming Yan;Xiu Yang;Q. Ye

Implicit Graph Neural Networks: A Monotone Operator Viewpoint

• 发表时间: 2023
• 作者: Justin Baker;Qingsong Wang;C. Hauck;Bao Wang

Learning Proper Orthogonal Decomposition of Complex Dynamics Using Heavy-ball Neural ODEs

使用重球神经常微分方程学习复杂动力学的正确正交分解

• DOI: 10.1007/s10915-023-02176-8
• 发表时间: 2023
• 期刊: Journal of Scientific Computing
• 影响因子: 2.5
• 作者: Baker, Justin;Cherkaev, Elena;Narayan, Akil;Wang, Bao
• 通讯作者: Wang, Bao