RI: Small: Non-parametric Machine Learning in the Age of Deep and High-Dimensional Models

RI：小：深度和高维模型时代的非参数机器学习

• 批准号: 1909816
• 负责人: Pradeep Ravikumar
• 金额: $ 44.99万
• 依托单位: Carnegie-Mellon University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-10-01 至 2023-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1909816&HistoricalAwards=false
• 关键词: RI; Small; Non; parametric; Machine; RI; Small; Non; parametric; Machine

The empirical successes of machine learning in recent years are due, in a large part, to models that are highly flexible (such as deep neural networks), which can tackle complex tasks (such as predicting whether a medical image is indicative of cancer). These models learn from large datasets and require a lot of empirical engineering expertise. The current belief is that a better understanding of this engineering practice might help us gain insights into the relationship between the models and the sample size of the datasets. Taking inspirations for deep neural networks, the research in this project formulates broader classes of models and algorithms that learn from fewer samples but do not require complex engineering expertise. The research enables learning highly flexible models in a more rigorous and reproducible manner; consequently, the users may have greater trust in the resulting applications.More specifically, the research in this project leverages insights from deep and high-dimensional models to develop a new class of non-parametric prediction as well as density functions. The project develops novel extensions of parametric structural sparsity constraints to the non-parametric estimation setting. By treating the multivariate prediction functions as functional generalizations of tensors, the project develops novel extensions of structural sparsity constraints designed for parametric model parameters to novel counterparts for prediction functions. The project also investigates a "destructive learning" approach to learning deep compositional models, which have a similar compositional form to deep neural network models. The project develops stage-wise algorithms to learn such deep compositional models, similar to boosting, by iteratively finding and destroying information in the data using well-studied shallow learning algorithms in each stage.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

近年来，机器学习的经验成功在很大程度上是由于高度灵活的模型（例如深神经网络），这些模型可以解决复杂的任务（例如预测医学图像是否表示癌症）。这些模型从大型数据集中学习，并需要大量的经验工程专业知识。当前的信念是，更好地了解这种工程实践可能有助于我们深入了解模型与数据集样本之间的关系。该项目的研究汲取了深层神经网络的灵感，为更少的模型和算法制定了更广泛的模型和算法，这些模型和算法从较少的样本中学习但不需要复杂的工程专业知识。该研究使学习高度灵活的模型以更严格和可重复的方式学习。因此，用户可能对由此产生的应用程序有更大的信任。更具体地说，该项目的研究利用了深层和高维模型的见解来开发一类新的非参数预测以及密度功能。该项目将参数结构稀疏性约束的新型扩展扩展到非参数估计设置。通过将多元预测视为张量的功能概括，该项目开发了针对参数模型参数设计的结构稀疏性约束的新型扩展，以实现用于预测功能的新型对应物。该项目还研究了一种学习深层组成模型的“破坏性学习”方法，该方法具有与深神经网络模型相似的组成形式。该项目通过迭代地查找和破坏数据中的信息，在每个阶段都在数据中找到和破坏数据中的信息，从而开发了阶段的算法，以学习和破坏数据中的信息。该奖项反映了NSF的法定任务，并通过使用基金会的知识分子优点和广泛的影响来评估NSF的法定任务。