Collaborative Research: Foundations of Deep Learning: Theory, Robustness, and the Brain

协作研究：深度学习的基础：理论、稳健性和大脑 —

基本信息

批准号：
2134108
负责人：
Tomaso Poggio
金额：
$ 60万
依托单位：
Massachusetts Institute of Technology
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2021
资助国家：
美国
起止时间：
2021-12-01 至 2024-11-30
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=2134108&HistoricalAwards=false
关键词：
Collaborative Research Foundations Deep Learning

项目摘要

A truly comprehensive theory of machine learning has the potential of informing science and engineering in the same profound way Maxwell’s equations did. It was the development of that theory by Maxwell that truly unleashed the potential of electricity, leading to radio, radars, computers, and the Internet. In an analogy, deep learning (DL) has found over the past decade many applications, so far without a comprehensive theory. An eventual theory of learning that explains why and how deep networks work and what their limitations are may thus enable the development of even more powerful learning approaches – especially if the goal of reconnecting DL to brain research bears fruit. In the long term, the ability to develop and build better intelligent machines will be essential to any technology-based economy. After all, even in its current – still highly imperfect – state, DL is impacting or about to impact just about every aspect of our society and life. The investigators also plan to complement their theoretical research with the educational goal of training a diverse population of young researchers from mathematics, computer science, statistics, electrical engineering, and computational neuroscience in the field of machine learning and of its theoretical underpinnings.The investigators propose to join forces in a multi-pronged and collaborative assault on the profound mysteries of DL, informed by the sum of their experience, expertise, ideas, and insight. The research goals are threefold: to develop a sound foundational/mathematical understanding of DL; in doing so to advance the foundational understanding of learning more generally; and to advance the practice of DL by addressing its above-mentioned weaknesses. Of six foundational thrusts, the first two focus on the standard decomposition of the prediction error in approximation and sample (or estimation) error. Their goal is to extend classical results in approximation theory and theory of learnability to DL. These two are then supported by a research project that is specific to deep learning: analysis of the dynamics of gradient descent in training a network. The fourth theme is about robustness against adversaries and shifts, a powerful test for theories which is also important for practical deployment of learning systems. The fifth thrust is about developing the theory of control through DL, as well as exploring dynamical systems aspects of deep reinforcement learning. The final topic connects research on DL to its origins - and possibly its future: networks of neurons in the brain. The proposed research also promises to advance the foundations of learning theory. Success in this project will result in sharper mathematical techniques for machine learning and comprehensive foundations of machine learning robustness, broadly construed. It will also ultimately enable development of learning algorithms that transcend deep learning and guide the way towards creating more intelligent machines, and shed new light on our own intelligence.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

真正全面的机器学习理论具有以与麦克斯韦方程相同的深刻方式为科学和工程提供信息的潜力。麦克斯韦对该理论的发展真正释放了电力的潜力，导致了无线电、雷达、计算机和电力的发展。打个比方，深度学习（DL）在过去十年中已经有了许多应用，但迄今为止还没有一个全面的理论来解释深度网络为何、如何工作以及它们的局限性。开发更强大的学习方法 –尤其是如果将深度学习与高度大脑研究重新联系起来的目标从长远来看，开发和建造更好的智能机器的能力对于任何基于技术的经济都至关重要，即使在目前仍然不完善。 – 状态，DL 正在影响或即将影响我们社会和生活的几乎各个方面，研究人员还计划通过培训数学、计算机科学、统计学、电气等领域的不同年轻研究人员的教育目标来补充他们的理论研究。机器学习及其领域的工程和计算神经科学研究人员建议结合他们的经验、专业知识、想法和洞察力，联合起来对深度学习的奥秘进行多管齐下的协作攻击。研究目标有三个：建立健全的基础。 /对深度学习的数学理解；这样做是为了更普遍地促进对学习的基本理解；并通过解决深度学习的上述弱点，其中前两个重点是标准。他们的目标是将近似理论和可学习性理论的经典结果扩展到深度学习，然后由一个特定于深度学习的研究项目支持：分析。第四个主题是关于对抗对手和变化的鲁棒性，这是对理论的有力检验，这对于学习系统的实际部署也很重要。第五个主题是关于通过深度学习发展控制理论。还有最后一个主题将深度强化学习的研究与它的起源联系起来——可能还有它的未来：大脑中的神经元网络。这项研究也有望为该项目的成功奠定基础。将为机器学习带来更敏锐的数学技术和机器学习鲁棒性的全面基础，从广义上讲，它还将最终实现超越深度学习的学习算法的开发，并指导创建更智能的机器的道路，并为我们自己带来新的启示。这个奖项体现了通过使用基金会的智力价值和更广泛的影响审查标准进行评估，NSF 的法定使命被认为值得支持。

项目成果

期刊论文数量（16）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

STay-ON-the-Ridge: Guaranteed Convergence to Local Minimax Equilibrium in Nonconvex-Nonconcave Games

DOI：
10.48550/arxiv.2210.09769
发表时间：
2022-10
期刊：
影响因子：
0
作者：
C. Daskalakis;Noah Golowich;Stratis Skoulakis;Manolis Zampetakis
通讯作者：
C. Daskalakis;Noah Golowich;Stratis Skoulakis;Manolis Zampetakis

The Complexity of Markov Equilibrium in Stochastic Games

随机博弈中马尔可夫均衡的复杂性

DOI：
发表时间：
2023
期刊：
Proceedings of the 36th Annual Conference on Learning Theory (COLT
影响因子：
0
作者：
Constantinos Daskalakis;Noah Golowich;Kaiqing Zhang
通讯作者：
Kaiqing Zhang

The Complexity of Markov Equilibrium in Stochastic Games.

随机博弈中马尔可夫均衡的复杂性。

DOI：
发表时间：
2023
期刊：
36th Annual Conference on Learning Theory (COLT
影响因子：
0
作者：
Constantinos Daskalakis;Noah Golowich;Kaiqing Zhang
通讯作者：
Kaiqing Zhang

Sign and Basis Invariant Networks for Spectral Graph Representation Learning

DOI：
发表时间：
2022-02
期刊：
ArXiv
影响因子：
0
作者：
Derek Lim;Joshua Robinson;Lingxiao Zhao;T. Smidt;S. Sra;Haggai Maron;S. Jegelka
通讯作者：
Derek Lim;Joshua Robinson;Lingxiao Zhao;T. Smidt;S. Sra;Haggai Maron;S. Jegelka

Online Learning and Solving Infinite Games with an ERM Oracle.

使用 ERM Oracle 在线学习和解决无限游戏。

DOI：
发表时间：
2023
期刊：
Proceedings of the 36th Annual Conference on Learning Theory (COLT
影响因子：
0
作者：
Angelos Assos;Idan Attias;Yuval Dagan;Constantinos Daskalakis;Maxwell Fishelson
通讯作者：
Maxwell Fishelson