CAREER: Scaling Up Knowledge Discovery in High-Dimensional Data Via Nonconvex Statistical Optimization

职业：通过非凸统计优化扩大高维数据中的知识发现

• 批准号: 1906169
• 负责人: Quanquan Gu
• 金额: $ 50.6万
• 依托单位: University of California-Los Angeles
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-07-01 至 2024-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1906169&HistoricalAwards=false
• 关键词: CAREER; Scaling; Up; Knowledge; Discovery

The past decade has witnessed a surge of research activities on knowledge discovery in high-dimensional data, among which convex optimization-based methods are widely used. While convex optimization algorithms enjoy global convergence guarantees, they are not always scalable to high-dimensional massive data. Motivated by the empirical success of nonconvex methods such as matrix factorization, the objective of this project is to develop a new generation of principled nonconvex statistical optimization algorithms to scale up high-dimensional machine learning methods. This project amplifies the utility of high-dimensional knowledge discovery methods in various fields such as computational genomics and recommendation systems. It incorporates the resulting research outcomes into curriculum development and online courses, to train a new generation of machine learning and data mining practitioners. In addition, special training is provided to K-12 students and community college students for a broader education of modern data analysis techniques.This project consists of three synergistic research thrusts. First, it develops a family of nonconvex algorithms for structured sparse learning, including extensions to both parallel computing and distributed computing. Second, it devises a unified nonconvex optimization framework for low-rank matrix estimation, which covers a wide range of low-rank matrix learning problems such as matrix completion and preference learning. Several acceleration techniques are also explored. Third, it develops a family of alternating optimization algorithms, to solve the bi-convex optimization problem for estimating various complex statistical models. This project integrates modern optimization techniques with model-based statistical thinking, and provides a systematic way to design nonconvex high-dimensional machine learning methods with strong theoretical guarantees. The targeted applications include but not limited to computational genomics, neuroscience, and recommendation systems.

在过去的十年中，在高维数据中见证了有关知识发现的大量研究活动，其中广泛使用了基于凸优化的方法。虽然凸优化算法享有全球收敛的保证，但它们并不总是可扩展到高维大量数据。由非凸方法（例如矩阵分解）的经验成功所激发，该项目的目的是开发新一代的原则性非凸统计优化算法，以扩展高维机器学习方法。该项目放大了在各个领域（例如计算基因组学和推荐系统）中高维知识发现方法的实用性。它将结果的研究成果纳入了课程开发和在线课程中，以培训新一代的机器学习和数据挖掘从业人员。此外，还向K-12学生和社区大学生提供了对现代数据分析技术的更广泛教育的特殊培训。该项目由三个协同的研究推力组成。首先，它为结构化稀疏学习开发了一个非凸算法的家族，包括对并行计算和分布式计算的扩展。其次，它为低级别矩阵估计设计了一个统一的非convex优化框架，该框架涵盖了各种低级别矩阵学习问题，例如矩阵完成和偏好学习。还探索了几种加速技术。第三，它开发了一个交替优化算法的家族，以解决估计各种复杂统计模型的BI-CONVEX优化问题。该项目将现代优化技术与基于模型的统计思维集成在一起，并提供了一种系统的方法来设计非凸高维机器学习方法，并具有强大的理论保证。目标应用包括但不限于计算基因组学，神经科学和推荐系统。

项目成果

On the Global Convergence of Training Deep Linear ResNets

• 发表时间: 2020-03
• 期刊: ArXiv
• 作者: Difan Zou;Philip M. Long;Quanquan Gu

A Simple and Provably Efficient Algorithm for Asynchronous Federated Contextual Linear Bandits

• DOI: 10.48550/arxiv.2207.03106
• 发表时间: 2022-07
• 期刊: ArXiv
• 作者: Jiafan He;Tianhao Wang;Yifei Min;Quanquan Gu

Accelerated Factored Gradient Descent for Low-Rank Matrix Factorization

• 发表时间: 2020
• 作者: Dongruo Zhou;Yuan Cao;Quanquan Gu

Sampling from Non-Log-Concave Distributions via Variance-Reduced Gradient Langevin Dynamics

• 发表时间: 2019-04
• 作者: Difan Zou;Pan Xu;Quanquan Gu

A Generalized Neural Tangent Kernel Analysis for Two-layer Neural Networks

• 发表时间: 2020-02
• 期刊: arXiv: Learning
• 作者: Zixiang Chen;Yuan Cao;Quanquan Gu;Tong Zhang