Collaborative Research: Differential Equations Motivated Multi-Agent Sequential Deep Learning: Algorithms, Theory, and Validation

协作研究：微分方程驱动的多智能体序列深度学习：算法、理论和验证

• 批准号: 2152762
• 负责人: Bao Wang
• 金额: $ 10万
• 依托单位: University of Utah
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-07-01 至 2025-06-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2152762&HistoricalAwards=false
• 关键词: Collaborative; Research; Differential; Equations; Motivated

Sequential data observed from multiple agents is ubiquitous in artificial intelligence (AI) and scientific applications, for example, in computer vision, natural language processing, robotics, computational biology and biophysics, and knowledge graphs. Learning from sequentially observed data often provides a global understanding of the underlying system and yields more reliable predictions than learning from a non-sequentially (single-shot) observed data. Sequential data is often irregularly-sampled in time and space and when this is combined with the interaction between agents, it raises tremendous challenges for machine learning. This project addresses these challenges by developing new mathematical understandings of these bottlenecks combined with new mathematically-principled deep learning algorithms for sequential and graph learning. Anticipated results and algorithms from this project will have broad applicability to important societal issues, such as pandemic spread, cooperative robotics, and environmental change. The project includes research training opportunities for graduate students.This project bridges ordinary differential equations (ODEs) and partial differential equations (PDEs) theory with multi-agent sequential learning practice. The project further leverages ODE and PDE insights to advance theoretically-grounded algorithms for deep sequential and graph learning. This project synergistically integrates recent advances in neural ODE methods with recent advances in graph networks for machine learning. The project develops and explores building next-generation algorithms based on wave equations on graphs, coupling second-order continuous dynamics in time with graph filtering. The research includes theoretical guarantees for the new methods in overcoming the over-smoothing issue, to enable sequential learning on graphs with deep architectures.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

从多个代理观察到的序列数据在人工智能 (AI) 和科学应用中无处不在，例如计算机视觉、自然语言处理、机器人、计算生物学和生物物理学以及知识图谱。从顺序观察的数据中学习通常可以提供对底层系统的全局理解，并比从非顺序（单次）观察的数据中学习产生更可靠的预测。序列数据通常在时间和空间上不规则采样，当这与代理之间的交互相结合时，它给机器学习带来了巨大的挑战。该项目通过对这些瓶颈提出新的数学理解，并结合用于顺序学习和图形学习的新的数学原理深度学习算法来解决这些挑战。该项目的预期结果和算法将广泛适用于重要的社会问题，例如流行病传播、协作​​机器人和环境变化。该项目包括为研究生提供研究培训机会。该项目将常微分方程 (ODE) 和偏微分方程 (PDE) 理论与多智能体顺序学习实践联系起来。该项目进一步利用 ODE 和 PDE 见解来推进基于理论的深度序列和图形学习算法。该项目协同地将神经 ODE 方法的最新进展与机器学习图网络的最新进展相结合。该项目开发并探索基于图波动方程构建下一代算法，将二阶连续动力学及时与图滤波耦合。 该研究为克服过度平滑问题的新方法提供了理论保证，从而能够在具有深层架构的图上进行顺序学习。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响进行评估，被认为值得支持审查标准。

项目成果

Implicit Graph Neural Networks: A Monotone Operator Viewpoint

隐式图神经网络：单调算子观点

• 发表时间: 2023-07
• 期刊: International Conference on Machine Learning
• 作者: Justin Baker; Qingsong Wang
• 通讯作者: Qingsong Wang

Efficient and Reliable Overlay Networks for Decentralized Federated Learning

用于去中心化联邦学习的高效可靠的覆盖网络

• DOI: 10.1137/21m1465081
• 发表时间: 2022-08
• 期刊: SIAM Journal on Applied Mathematics
• 影响因子: 1.9
• 作者: Hua, Yifan;Miller, Kevin;Bertozzi, Andrea L.;Qian, Chen;Wang, Bao
• 通讯作者: Wang, Bao

How does momentum benefit deep neural networks architecture design? A few case studies

动量如何有利于深度神经网络架构设计？

• DOI: 10.1007/s40687-022-00352-0
• 发表时间: 2022-09
• 期刊: Research in the Mathematical Sciences
• 影响因子: 1.2
• 作者: Wang, Bao;Xia, Hedi;Nguyen, Tan;Osher, Stanley
• 通讯作者: Osher, Stanley

Learning Proper Orthogonal Decomposition of Complex Dynamics Using Heavy-ball Neural ODEs

使用重球神经常微分方程学习复杂动力学的正确正交分解

• DOI: 10.1007/s10915-023-02176-8
• 发表时间: 2023-05
• 期刊: Journal of Scientific Computing
• 影响因子: 2.5
• 作者: Baker, Justin;Cherkaev, Elena;Narayan, Akil;Wang, Bao
• 通讯作者: Wang, Bao

Improving Deep Neural Networks’ Training for Image Classification With Nonlinear Conjugate Gradient-Style Adaptive Momentum

使用非线性共轭梯度式自适应动量改进深度神经网络 — 图像分类训练

• DOI: 10.1109/tnnls.2023.3255783
• 发表时间: 2023-03
• 期刊: IEEE Transactions on Neural Networks and Learning Systems
• 影响因子: 10.4
• 作者: Wang, Bao;Ye, Qiang
• 通讯作者: Ye, Qiang