CAREER: Foundations of Resource Efficient Machine Learning

职业：资源高效机器学习的基础

• 批准号: 2046816
• 负责人: Samet Oymak
• 金额: $ 55.9万
• 依托单位: University of California-Riverside
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-02-01 至 2026-01-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2046816&HistoricalAwards=false
• 关键词: CAREER; Foundations; Resource; Efficient; Machine

Contemporary machine learning techniques tend to be resource-intensive, often requiring good quality datasets, expensive hardware, or significant computing power. In a wide array of application domains, ranging from healthcare to mobile computing, these critical resources are lacking. Novel methodologies that enable the optimal utilization of resources can help unlock the full potential of the data science revolution for these domains. Towards this aim, this project will develop theoretically-grounded algorithms to facilitate the design of machine learning models under application-specific resource constraints. The outcomes of the project will help enable machine learning methods to operate with less human-annotated data, less computing power, and on a wider range of hardware platforms. To demonstrate interdisciplinary impact, the resulting algorithms will be employed in the design of efficient hydrological models which aid in predicting and managing water resources. The research will also be strongly coupled with education through the mentoring of undergraduate students, new undergraduate and graduate course development, and live broadcasts of the lectures over publicly accessible online platforms.This project aims to develop the foundational theories and algorithms to guide the efficient use of statistical and computational resources. The research on the statistical front focuses on the data and will uncover the fundamental tradeoffs between the data amount, label quality, and the model accuracy. Understanding these tradeoffs will lead to the design of improved loss functions and regularization techniques. On the computational front, theory-inspired model compression schemes will be developed by exploring the interplay between the model size and accuracy. Secondly, the model performance will be enhanced by identifying the optimal model architecture via computationally-efficient algorithms that co-design the architecture, compression scheme, and the loss function. These theoretical and algorithmic investigations will utilize tools from statistical learning, optimization, deep learning theory, and high-dimensional probability. The proposed research is expected to provide much-needed theoretical basis for poorly-understood heuristics in fields spanning semi-supervised learning, model compression, neural architecture search, and will guide the design of next-generation algorithms achieving the optimal resource tradeoffs.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

当代机器学习技术往往是资源密集型的，通常需要高质量的数据集、昂贵的硬件或强大的计算能力。从医疗保健到移动计算等广泛的应用领域都缺乏这些关键资源。实现资源最佳利用的新颖方法可以帮助释放这些领域数据科学革命的全部潜力。为了实现这一目标，该项目将开发基于理论的算法，以促进在特定应用资源限制下的机器学习模型的设计。该项目的成果将有助于使机器学习方法能够在更少的人工注释数据、更少的计算能力和更广泛的硬件平台上运行。为了展示跨学科影响，所得算法将用于设计有效的水文模型，以帮助预测和管理水资源。该研究还将通过本科生指导、新的本科生和研究生课程开发以及在公共在线平台上直播讲座，与教育紧密结合。该项目旨在开发基础理论和算法来指导高效使用统计和计算资源。统计前沿的研究重点是数据，并将揭示数据量、标签质量和模型准确性之间的基本权衡。了解这些权衡将有助于设计改进的损失函数和正则化技术。在计算方面，将通过探索模型大小和精度之间的相互作用来开发受理论启发的模型压缩方案。其次，通过共同设计架构、压缩方案和损失函数的计算高效算法来确定最佳模型架构，从而提高模型性能。这些理论和算法研究将利用统计学习、优化、深度学习理论和高维概率的工具。所提出的研究有望为半监督学习、模型压缩、神经架构搜索等领域中鲜为人知的启发式方法提供急需的理论基础，并将指导下一代算法的设计，实现最佳资源权衡。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Max-Margin Token Selection in Attention Mechanism

• DOI: 10.48550/arxiv.2306.13596
• 发表时间: 2023-06
• 期刊: ArXiv
• 作者: Davoud Ataee Tarzanagh;Yingcong Li;Xuechen Zhang;Samet Oymak

Class-attribute Priors: Adapting Optimization to Heterogeneity and Fairness Objective

• DOI: 10.48550/arxiv.2401.14343
• 发表时间: 2024-01
• 作者: Xuechen Zhang;Mingchen Li;Jiasi Chen;Christos Thrampoulidis;Samet Oymak

A Theoretical Characterization of Semi-supervised Learning with Self-training for Gaussian Mixture Models

• 发表时间: 2021
• 作者: Samet Oymak;Talha Cihad Gulcu

Revisiting Ho-Kalman based system identification: robustness and finite-sample analysis

重新审视基于 Ho-Kalman 的系统识别：鲁棒性和有限样本分析

• DOI: 10.1109/tac.2021.3083651
• 发表时间: 2021
• 期刊: IEEE transactions on automatic control
• 影响因子: 6.8
• 作者: Oymak, Samet;Ozay, Necmiye
• 通讯作者: Ozay, Necmiye

Non-asymptotic and Accurate Learning of Nonlinear Dynamical Systems

• 发表时间: 2020-02
• 期刊: J. Mach. Learn. Res.
• 作者: Yahya Sattar;Samet Oymak