Collaborative Research: CIF: Small: Deep Sparse Models: Analysis and Algorithms

合作研究：CIF：小型：深度稀疏模型：分析和算法

• 批准号: 2240708
• 负责人: Zhihui Zhu
• 金额: $ 20.55万
• 依托单位: Ohio State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-08-01 至 2024-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2240708&HistoricalAwards=false
• 关键词: Collaborative; Research; CIF; Small; Deep

Deep convolutional neural networks are a class of mathematical models that provide a variety of machine learning tools with impressive success, often obtaining state-of-the-art results across different fields. Yet, their theoretical understanding and the fundamental ideas behind these algorithms have remained elusive. These questions are essential to recognize and characterize their limitations, to provide guarantees for their performance, and even to develop and engineer improved practical models. A promising approach to obtain this understanding is to make assumptions about the class of samples on which these models are deployed (e.g., so that these are "simple enough") with the intention of providing theoretical insights about them. Further understanding of this 'multi-layered convolutional sparse model' is what this project seeks accomplish, broadening the understanding of its related optimization and learning problems, and shedding light on deep learning methodologies.This project proposes to advance the state of the art in generalized sparse models of different numbers of layers, focusing on both inference and learning problems. Provable and efficient optimization methods will be derived for the inverse problems associated with multilayer sparse models by relying on new results in proximal gradient and subgradient descent methods. This proposal will further extend the formulation of the pursuit to other settings, increasing stability and robustness to the choice of parameters and to outliers. Furthermore, efficient algorithms for the corresponding unsupervised learning problem will be proposed and analyzed. Questions of sample complexity and generalization bounds will in turn be studied in supervised learning settings. Throughout this project, the resulting algorithms will be studied in terms of their relation to specific convolutional network architectures. The project brings together combined expertise in signal processing, dictionary learning, machine learning, and the design, analysis and implementation of optimization methods for large-scale problems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

深度卷积神经网络是一类数学模型，可提供具有令人印象深刻成功的各种机器学习工具，通常会在不同领域获得最先进的结果。然而，他们的理论理解和这些算法背后的基本思想仍然难以捉摸。这些问题对于认识和表征其局限性，为其绩效提供保证，甚至开发和设计改进实际模型至关重要。获得这种理解的一种有希望的方法是对这些模型的部署类别（例如，使这些模型都“足够简单”）做出假设，目的是提供有关它们的理论见解。该项目寻求完成的“多层卷积稀疏模型”的进一步了解，扩大了对其相关优化和学习问题的理解，并阐明了深度学习方法。该项目提议在不同数量的层次的广义稀疏模型中推进艺术的状态，从而侧重于选择和学习问题。通过依靠近端梯度和亚速度下降方法中的新结果，将为与多层稀疏模型相关的反问题提供可证明的优化方法。该提案将进一步将追求的表述扩展到其他环境，增加对参数和异常值的稳定性和鲁棒性。此外，将提出和分析相应的无监督学习问题的有效算法。样本复杂性和泛化范围的问题又将在监督的学习设置中进行研究。在整个项目中，将根据其与特定的卷积网络体系结构的关系来研究所得的算法。该项目汇集了在信号处理，词典学习，机器学习以及针对大规模问题的优化方法的设计，分析和实施方面的综合专业知识。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的知识分子优点和更广泛的影响审查标准通过评估来进行评估的。

项目成果

OTOV2: Automatic, Generic, User-Friendly

• DOI: 10.48550/arxiv.2303.06862
• 发表时间: 2023-03
• 期刊: ArXiv
• 作者: Tianyi Chen;Luming Liang;Tian Ding;Zhihui Zhu;Ilya Zharkov

Neural Collapse with Normalized Features: A Geometric Analysis over the Riemannian Manifold

• DOI: 10.48550/arxiv.2209.09211
• 发表时间: 2022-09
• 期刊: ArXiv
• 作者: Can Yaras;Peng Wang;Zhihui Zhu;L. Balzano;Qing Qu

Are All Losses Created Equal: A Neural Collapse Perspective

• DOI: 10.48550/arxiv.2210.02192
• 发表时间: 2022-10
• 期刊: ArXiv
• 作者: Jinxin Zhou-;Chong You;Xiao Li;Kangning Liu;Sheng Liu;Qing Qu;Zhihui Zhu

Revisiting Sparse Convolutional Model for Visual Recognition

• DOI: 10.48550/arxiv.2210.12945
• 发表时间: 2022-10
• 期刊: ArXiv
• 作者: Xili Dai;Mingyang Li;Pengyuan Zhai;Shengbang Tong;Xingjian Gao;Shao-Lun Huang;Zhihui Zhu;Chong You;Y. Ma