Collaborative Research: SCALE MoDL: Representation Theoretic Foundations of Deep Learning

合作研究：SCALE MoDL：深度学习的表示理论基础

• 批准号: 2134274
• 负责人: Qi Yu
• 金额: $ 30万
• 依托单位: University of California-San Diego
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-01-01 至 2024-12-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2134274&HistoricalAwards=false
• 关键词: Collaborative; Research; SCALE; MoDL; Representation; Collaborative; Research; SCALE; MoDL; Representation

In the past decade, deep learning has had transformative impacts across society. However, progress has often relied on heuristic methods, massive data, and great computing power. This comes with limited theoretical understanding and has at times given rise to failures of generalization and vulnerable performance in extreme scenarios. This project will address these limitations by developing strong theoretical foundations for deep learning using representation theory, which is the mathematical study of symmetry. Symmetry plays a key role in human reasoning. Greater understanding of the role symmetry plays in deep learning will unlock a variety of improved models. These include models that can learn from scientific knowledge and not just raw data, models with trustable, guaranteed performance, and models that can recombine patterns they have already learned — as humans do easily — to generalize to new situations more rapidly. An explicit goal of this project is to broaden research into why deep learning works. To this end, the investigators will integrate the research into education and establish a mentorship program for high school students from groups underrepresented in science.The goal of the research is to understand the role of representation theory in enabling efficient optimization and improved generalization of deep learning even in domains with approximate or unknown symmetry. This project pursues three lines of research that will broaden the impact of representation theory in deep learning beyond strict inductive biases. The first is the trade-off between the degree of symmetry in the model and the degree of symmetry in the domain. This line of research will study networks that combine equivariant and non-equivariant features. The second line of research will examine learning symmetry directly from data to improve generalization in domains without known symmetries. The third aim is to develop a theoretical basis for deep learning using quiver representations. This perspective reveals the symmetry of the structure of deep-learning models themselves, through their parameter spaces, even when the domains have no obvious symmetry.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

在过去的十年中，深度学习在整个社会上产生了变革性的影响。但是，进步通常依赖于启发式方法，大量数据和出色的计算能力。这是有限的理论理解，有时会导致在极端情况下的概括和脆弱性能的失败。该项目将通过使用代表理论为对称性的数学研究开发强大的理论基础来解决这些局限性。对称性在人类推理中起关键作用。对对称性在深度学习中所起的作用有更多的了解将解锁各种改进的模型。这些模型包括可以从科学知识中学习的模型，而不仅仅是原始数据，具有可信赖性，保证性能的模型，以及可以重新组合他们已经学到的模式（就像人类一样容易）的模型，以更快地推广到新情况。该项目的明确目标是扩大研究深度学习作用的研究。为此，研究人员将将研究研究整合到教育中，并为科学领域不足的小组的高中学生建立心态计划。该研究的目的是了解代表理论在实现有效优化和改善深度学习的作用，即使在具有近似或未知对称性的领域的深度学习中，也是如此。该项目追求三项研究，将扩大代表理论在深度学习中的影响，超越严格的归纳偏见。首先是模型中对称程度与域中对称程度之间的权衡。这一研究将研究结合均等和非等价特征的网络。第二条研究将直接从数据中检查学习对称性，以改善没有已知对称性的域中的概括。第三个目的是建立使用颤抖表征深度学习的理论基础。这种观点通过参数空间揭示了深度学习模型本身的结构的对称性，即使域没有明显的对称性，该奖项反映了NSF的法定任务，并被认为是通过基金会的智力优点和更广泛的影响审查标准通过评估来评估的。