Collaborative Research: CIF: Medium: Foundations of Robust Deep Learning via Data Geometry and Dyadic Structure

合作研究：CIF：媒介：通过数据几何和二元结构实现稳健深度学习的基础

基本信息

批准号：
2212326
负责人：
Qing Qu
金额：
$ 47.2万
依托单位：
Regents of the University of Michigan - Ann Arbor
依托单位国家：
美国
项目类别：
Continuing Grant
财政年份：
2022
资助国家：
美国
起止时间：
2022-10-01 至 2026-09-30
项目状态：
未结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=2212326&HistoricalAwards=false
关键词：
Collaborative Research CIF Medium Foundations

项目摘要

Deep learning is a subfield of machine learning that uses artificial neural networks to learn patterns from data. Deep learning methods currently achieve the best performance in important problems such as facial recognition, image processing, speech recognition, and machine translation and have also been used in scientific problems in space weather, molecular biology, chemistry, and the health sciences. Despite this success, deep-learning methods are still widely viewed as black boxes, where their decisions are difficult to interpret. Furthermore, there is little mathematical understanding about how to design an effective deep neural network. This project will advance this understanding by investigating how the geometry of the data can be used to better understand and improve deep-learning design strategies. These advances will lead to improved performance in deep learning in scientific fields with large societal impact such as medicine, public health, molecular biology, and chemistry. The research project also involves the mentoring of doctoral students and the material generated from the project will be incorporated into multiple courses on data science. In this project, the team of researchers will develop a comprehensive theoretical framework for harnessing data geometry, graphs, and invariance to enhance performance and interpretability for a wide scope of deep learning problems. This will include investigating the effects of neural network transformations on the data geometry, which is captured by applying manifold learning at different layers in the neural network. New metrics will be designed to empirically test invariance relative to a symmetry group at different layers. These metrics will be used to drive a data-augmentation process that maximizes the learned invariance of the neural network during training. Methods for detecting hidden invariances in data will also be developed that can be used to analyze the neural network training process. By investigating these problems, this project will provide deep-neural-network design guidelines that will lead to a new generation of deep learning that moves beyond heuristics and ensures that a neural network is designed to ensure accuracy, robustness, and interpretability for the learning task.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

深度学习是机器学习的一个子领域，它使用人工神经网络从数据中学习模式。深度学习方法目前在面部识别、图像处理、语音识别和机器翻译等重要问题上取得了最佳性能，并且还被应用于空间天气、分子生物学、化学和健康科学等科学问题。尽管取得了如此成功，深度学习方法仍然被广泛视为黑匣子，其决策难以解释。此外，对于如何设计有效的深度神经网络，人们缺乏数学理解。该项目将通过研究如何使用数据的几何形状来更好地理解和改进深度学习设计策略，从而推进这种理解。这些进步将提高医学、公共卫生、分子生物学和化学等具有重大社会影响的科学领域的深度学习性能。该研究项目还涉及对博士生的指导，该项目产生的材料将被纳入多个数据科学课程中。在该项目中，研究人员团队将开发一个全面的理论框架，用于利用数据几何、图形和不变性来增强广泛的深度学习问题的性能和可解释性。这将包括研究神经网络变换对数据几何的影响，这是通过在神经网络的不同层应用流形学习来捕获的。新的度量将被设计来凭经验测试相对于不同层的对称组的不变性。这些指标将用于驱动数据增强过程，最大限度地提高神经网络在训练过程中学到的不变性。还将开发用于检测数据中隐藏不变性的方法，可用于分析神经网络训练过程。通过研究这些问题，该项目将提供深度神经网络设计指南，这将导致新一代深度学习超越启发式，并确保神经网络的设计能够确保学习任务的准确性、鲁棒性和可解释性该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

期刊论文数量（5）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

Are All Losses Created Equal: A Neural Collapse Perspective

所有损失都是平等的吗：神经崩溃的观点

DOI：
发表时间：
2022-01
期刊：
Advances in neural information processing systems
影响因子：
0
作者：
Zhou, Jinxin;You, Chong;Li, Xiao;Liu, Kangning;Liu, Sheng;Qu, Qing;Zhu, Zhihui
通讯作者：
Zhu, Zhihui

Neural Collapse with Normalized Features: A Geometric Analysis over the Riemannian Manifold

具有归一化特征的神经崩溃：黎曼流形的几何分析