Collaborative Research: Differential Equations Motivated Multi-Agent Sequential Deep Learning: Algorithms, Theory, and Validation

协作研究：微分方程驱动的多智能体序列深度学习：算法、理论和验证

• 批准号: 2152762
• 负责人: Bao Wang
• 金额: $ 10万
• 依托单位: University of Utah
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-07-01 至 2025-06-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2152762&HistoricalAwards=false
• 关键词: Collaborative; Research; Differential; Equations; Motivated

Sequential data observed from multiple agents is ubiquitous in artificial intelligence (AI) and scientific applications, for example, in computer vision, natural language processing, robotics, computational biology and biophysics, and knowledge graphs. Learning from sequentially observed data often provides a global understanding of the underlying system and yields more reliable predictions than learning from a non-sequentially (single-shot) observed data. Sequential data is often irregularly-sampled in time and space and when this is combined with the interaction between agents, it raises tremendous challenges for machine learning. This project addresses these challenges by developing new mathematical understandings of these bottlenecks combined with new mathematically-principled deep learning algorithms for sequential and graph learning. Anticipated results and algorithms from this project will have broad applicability to important societal issues, such as pandemic spread, cooperative robotics, and environmental change. The project includes research training opportunities for graduate students.This project bridges ordinary differential equations (ODEs) and partial differential equations (PDEs) theory with multi-agent sequential learning practice. The project further leverages ODE and PDE insights to advance theoretically-grounded algorithms for deep sequential and graph learning. This project synergistically integrates recent advances in neural ODE methods with recent advances in graph networks for machine learning. The project develops and explores building next-generation algorithms based on wave equations on graphs, coupling second-order continuous dynamics in time with graph filtering. The research includes theoretical guarantees for the new methods in overcoming the over-smoothing issue, to enable sequential learning on graphs with deep architectures.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

从多个代理观察到的顺序数据在人工智能（AI）和科学应用中无处不在，例如在计算机视觉，自然语言处理，机器人技术，计算生物学和生物物理学以及知识图中。从依次观察到的数据中学习通常提供了对基础系统的全球理解，并产生比从非序列（单次）观察到的数据中获得更可靠的预测。顺序数据通常会在时间和空间中不规则地采样，当这种数据与代理之间的相互作用相结合时，它会给机器学习带来巨大的挑战。该项目通过对这些瓶颈进行新的数学理解，再加上新的数学原理深度学习算法来解决这些挑战，以进行顺序学习和图形学习。该项目的预期结果和算法将对重要的社会问题（例如大流行，合作机器人技术和环境变化）具有广泛的适用性。该项目包括为研究生提供的研究培训机会。该项目桥接了具有多代理顺序学习实践的普通微分方程（ODE）和部分微分方程（PDES）理论。该项目进一步利用ODE和PDE见解来推进理论上的算法，以进行深层顺序学习和图形学习。该项目协同整合了神经ode方法的最新进展与机器学习图网络的最新进展。该项目基于图形上的波方程来开发和探索构建下一代算法，并将二阶连续动态耦合到时间与图形滤波。 这项研究包括在克服过度平滑问题的新方法的理论保证中，以实现具有深度建筑的图表的顺序学习。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的知识分子优点评估来获得支持的，并具有更广泛的影响标准。

项目成果

Implicit Graph Neural Networks: A Monotone Operator Viewpoint

• 发表时间: 2023
• 作者: Justin Baker;Qingsong Wang;C. Hauck;Bao Wang

Efficient and Reliable Overlay Networks for Decentralized Federated Learning

• DOI: 10.1137/21m1465081
• 发表时间: 2021-12
• 期刊: SIAM J. Appl. Math.
• 作者: Yifan Hua;Kevin Miller;A. Bertozzi;Chen Qian;Bao Wang

Learning Proper Orthogonal Decomposition of Complex Dynamics Using Heavy-ball Neural ODEs

使用重球神经常微分方程学习复杂动力学的正确正交分解

• DOI: 10.1007/s10915-023-02176-8
• 发表时间: 2023
• 期刊: Journal of Scientific Computing
• 影响因子: 2.5
• 作者: Baker, Justin;Cherkaev, Elena;Narayan, Akil;Wang, Bao
• 通讯作者: Wang, Bao

How does momentum benefit deep neural networks architecture design? A few case studies

• DOI: 10.1007/s40687-022-00352-0
• 发表时间: 2021-10
• 期刊: Research in the Mathematical Sciences
• 影响因子: 1.2
• 作者: Bao Wang;Hedi Xia;T. Nguyen;S. Osher

Improving Deep Neural Networks’ Training for Image Classification With Nonlinear Conjugate Gradient-Style Adaptive Momentum

使用非线性共轭梯度式自适应动量改进深度神经网络 - 图像分类训练

• DOI: 10.1109/tnnls.2023.3255783
• 发表时间: 2023
• 期刊: IEEE Transactions on Neural Networks and Learning Systems
• 影响因子: 10.4
• 作者: Wang, Bao;Ye, Qiang
• 通讯作者: Ye, Qiang