CAREER: Statistical Analysis of Nonconvex Optimization in Unsupervised Learning

职业：无监督学习中非凸优化的统计分析

• 批准号: 1848575
• 负责人: Xiaodong Li
• 金额: $ 40.76万
• 依托单位: University of California-Davis
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-08-15 至 2024-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1848575&HistoricalAwards=false
• 关键词: CAREER; Statistical; Analysis; Nonconvex; Optimization

Unsupervised learning techniques have been widely used in real-world applications such as searching, ranking, recommender systems, social networks, online advertisements, online transportation, virtual assistants, and so on. In many practical applications of unsupervised learning methodology, nonconvex optimization based estimation methods are convenient due to their scalability to large data. This scalability is crucial in various applications such as computer vision and natural language processing. However, nonconvex optimization is computationally unstable or even infeasible in general due to unfavorable local minima that may exist. In consequence, statistical properties for global minimum based estimates may not provide meaningful guidelines for practitioners. This project aims to study the statistical properties of local minimum based estimates for nonconvex optimization methods in a range of unsupervised learning problems. The proposed research projects are significant in identifying reliable nonconvex frameworks by understanding the trade-off between computational feasibility and statistical efficiency. A new platform will be provided for collaborations across different fields such as statistics, mathematics and computer science. The activity is planned to engage female and underrepresented minority students in the study in Science, Technology, Engineering and Mathematics (STEM) fields through both theoretical and computational training and hands-on data analysis. Since nonconvex optimization methods are known to be adaptive to missing data and various parameterizations, the proposed projects are focused on the statistical analysis for low-rank factorization based unsupervised learning, such as matrix completion, robust PCA, pairwise ranking and network representation. Recent developments in landscape analysis for nonconvex low-rank factorization reveal that there could be no spurious local minima if (i) the ground truth satisfies strong structural assumptions; (ii) model mismatching is not permitted; (iii) the sample size is large. In contrast, the proposed project is focused on conducting the landscape analysis in more general settings: First, model-free frameworks will be proposed to study the geometric properties of nonconvex optimization without requiring structural assumptions on the data or exact model matching; Second, the effects of model mismatching on the landscape of the nonconvex objective functions will be analyzed; Third, statistical efficiencies for local minima based estimates and their relationship with the intrinsic dimensions and sample sizes will be established; Fourth, algebraic structures of general parameterized low-rank factorization will be exploited in order to establish a unified landscape analysis for a broad class of unsupervised learning problems. Moreover, in order to test the empirical behavior of the proposed methods, the activity is planned to identify appropriate benchmark datasets in learning-to-rank, predictive network analysis and recommendation systems in order to compare the empirical performances of the proposed approaches with baseline methods in the literature.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

无监督的学习技术已被广泛用于诸如搜索，排名，推荐系统，社交网络，在线广告，在线运输，虚拟助理等的现实应用程序中。在无监督学习方法的许多实际应用中，由于对大数据的可扩展性，基于非convex优化的估计方法很方便。这种可伸缩性在各种应用中至关重要，例如计算机视觉和自然语言处理。但是，由于可能存在的局部最小值，因此非convex优化在计算上是不稳定甚至不可行的。因此，基于全球最低估计的统计属性可能无法为从业者提供有意义的指南。该项目旨在研究在一系列无监督学习问题中，基于局部最小值的局部最低估计值的统计特性。拟议的研究项目对于通过了解计算可行性和统计效率之间的权衡而识别可靠的非Convex框架非常重要。将为跨不同领域的合作提供一个新平台，例如统计，数学和计算机科学。该活动计划通过理论和计算培训以及动手数据分析，使女性和代表性不足的少数族裔学生参与科学，技术，工程和数学（STEM）领域的研究。由于已知非convex优化方法适应缺失的数据和各种参数化，因此所提出的项目集中于基于低级别分解的无监督学习的统计分析，例如矩阵完成，鲁棒PCA，成对排名和网络表示。景观分析的最新进展是非凸率低分解的，表明如果（i）（i）地面真相满足强大的结构假设，则不会有虚假的局部最小值； （ii）不允许模型不匹配； （iii）样本量很大。相比之下，拟议项目的重点是在更一般的环境中进行景观分析：首先，将提出无模型的框架来研究非convex优化的几何特性，而无需在数据或确切的模型匹配上进行结构性假设；其次，将分析模型不匹配对非凸目标函数景观的影响；第三，将建立基于本地最小值的估计值及其与内在维度和样本量的关系；第四，将利用一般参数化低级分解的代数结构，以建立一系列无监督的学习问题的统一景观分析。此外，为了测试提议方法的经验行为，计划在学习到排名，预测网络分析和建议系统中确定适当的基准数据集，以比较所提出的方法的经验表现与文献中的基础方法的经验表现。本文的奖励反映了NSF的法定任务和经过评估，这是通过评估良好的依据。

项目成果

Nonconvex Matrix Completion with Linearly Parameterized Factors

• 发表时间: 2020-03
• 期刊: ArXiv
• 作者: Ji Chen;Xiaodong Li;Zongming Ma