CIF: Medium: Collaborative Research: Theory of Optimization Geometry and Algorithms for Neural Networks

CIF：媒介：协作研究：神经网络优化几何理论和算法

基本信息

批准号：
1900145
负责人：
Yingbin Liang
金额：
$ 40万
依托单位：
Ohio State University
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2019
资助国家：
美国
起止时间：
2019-10-01 至 2024-09-30
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1900145&HistoricalAwards=false
关键词：
CIF Medium Collaborative Research Theory

项目摘要

Deep learning has attracted a significant amount of interest in recent years due to its widespread applicability in computer vision, artificial intelligence and natural language processing, alongside recent strides in autonomous driving. The theoretical underpinnings behind such success, however, remain elusive to a large extent, hindering its further adoption in other applications. This project aims to advance the theoretical foundations of training neural networks in terms of optimization landscape and algorithmic efficacy, which in turn should have a measurable impact on the practice of deep learning by providing guiding principles for network design, algorithm selection, hyperparameter tuning, and adversarial training. This project adopts an interdisciplinary approach fusing ideas from machine learning, optimization, statistical signal processing, high-dimensional statistics, nonparametric statistics, and information theory. This project will likewise develop courses and tutorials on theoretical foundations of large-scale machine learning and provide extensive training opportunities for students at all levels.This project aims to develop a comprehensive theory to characterize the optimization landscape and geometry of loss functions and algorithmic regularizations of major neural network training problems, and explore how the network architecture---including depth, width, and activation functions---affect these properties, thus providing guidelines for the design of algorithms to train these networks more efficiently with theoretical performance guarantees. The project will explore the geometric properties and their impact on the optimization performance in training multi-layer neural networks, auto-encoders, generative adversarial networks, and adversarial training involving non-convex and saddle-point problems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

近年来，由于其在计算机视觉，人工智能和自然语言处理中的广泛适用性以及最近在自主驾驶方面的进步，深度学习引起了极大的兴趣。但是，这种成功背后的理论基础在很大程度上仍然难以捉摸，阻碍了其在其他应用中的进一步采用。该项目旨在从优化景观和算法效力方面推进训练神经网络的理论基础，这反过来又应通过为网络设计，算法选择，超级参数调整和对抗性培训提供指导性的指导原理来对深度学习实践产生可衡量的影响。该项目采用了一种跨学科的方法，从机器学习，优化，统计信号处理，高维统计，非参数统计和信息理论融合了思想。 This project will likewise develop courses and tutorials on theoretical foundations of large-scale machine learning and provide extensive training opportunities for students at all levels.This project aims to develop a comprehensive theory to characterize the optimization landscape and geometry of loss functions and algorithmic regularizations of major neural network training problems, and explore how the network architecture---including depth, width, and activation functions---affect these properties, thus providing设计算法设计指南，可以通过理论性能保证更有效地培训这些网络。该项目将探索几何特性及其对培训多层神经网络，自动编码器，生成对抗网络以及涉及非convex和鞍点问题的对抗性培训的影响的影响。该奖项反映了NSF的法定任务，并通过评估基金会的范围，反映了NSF的法定任务，并已被评估了基金会的范围。

项目成果

期刊论文数量（26）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

Improving Sample Complexity Bounds for (Natural) Actor-Critic Algorithms

DOI：
发表时间：
2020-04
期刊：
arXiv: Learning
影响因子：
0
作者：
Tengyu Xu;Zhe Wang-;Yingbin Liang
通讯作者：
Tengyu Xu;Zhe Wang-;Yingbin Liang

Deterministic policy gradient: Convergence analysis

DOI：
发表时间：
2022
期刊：
影响因子：
0
作者：
Huaqing Xiong;Tengyu Xu;Lin Zhao;Yingbin Liang;Wei Zhang
通讯作者：
Huaqing Xiong;Tengyu Xu;Lin Zhao;Yingbin Liang;Wei Zhang

History-Gradient Aided Batch Size Adaptation for Variance Reduced Algorithms

DOI：
发表时间：
2019-10
期刊：
影响因子：
0
作者：
Kaiyi Ji;Zhe Wang-;Bowen Weng;Yi Zhou;W. Zhang;Yingbin Liang
通讯作者：
Kaiyi Ji;Zhe Wang-;Bowen Weng;Yi Zhou;W. Zhang;Yingbin Liang

Convergence of Meta-Learning with Task-Specific Adaptation over Partial Parameters

DOI：
发表时间：
2020-06
期刊：
ArXiv
影响因子：
0
作者：
Kaiyi Ji;J. Lee;Yingbin Liang;H. Poor
通讯作者：
Kaiyi Ji;J. Lee;Yingbin Liang;H. Poor

Generalized-Smooth Nonconvex Optimization is As Efficient As Smooth Nonconvex Optimization

DOI：
发表时间：
2023-03
期刊：
影响因子：
0
作者：
Ziyi Chen;Yi Zhou;Yingbin Liang;Zhaosong Lu
通讯作者：
Ziyi Chen;Yi Zhou;Yingbin Liang;Zhaosong Lu

DOI：
{{ item.doi }}
发表时间：
{{ item.publish_year }}
期刊：
{{ item.journal_name }}
影响因子：
{{ item.factor }}
作者：
{{ item.authors }}
通讯作者：
{{ item.author }}

数据更新时间：{{ journalArticles.updateTime }}

作者：
{{ item.author }}

数据更新时间：{{ monograph.updateTime }}

作者：
{{ item.author }}

数据更新时间：{{ sciAawards.updateTime }}

作者：
{{ item.author }}

数据更新时间：{{ conferencePapers.updateTime }}

作者：
{{ item.author }}

数据更新时间：{{ patent.updateTime }}

Yingbin Liang其他文献

On the Equivalence of Two Achievable Regions for the Broadcast Channel

广播频道两个可达到区域的等效性

DOI：
10.1109/tit.2010.2090236
发表时间：
2011
期刊：
IEEE Transactions on Information Theory
影响因子：
2.5
作者：
Yingbin Liang;G. Kramer;H. Poor
通讯作者：
H. Poor

Capacity bounds for a class of cognitive interference channels with state

一类具有状态的认知干扰信道的容量界限

DOI：
10.1109/allerton.2011.6120223
发表时间：
2011
期刊：
2011 49th Annual Allerton Conference on Communication, Control, and Computing (Allerton)
影响因子：
0
作者：
Ruchen Duan;Yingbin Liang
通讯作者：
Yingbin Liang