Collaborative Research: AF: Small: A Unified Framework for Analyzing Adaptive Stochastic Optimization Methods Based on Probabilistic Oracles

合作研究：AF：Small：基于概率预言的自适应随机优化方法分析统一框架

基本信息

批准号：
2140057
负责人：
Katya Scheinberg
金额：
$ 25万
依托单位：
Cornell University
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2022
资助国家：
美国
起止时间：
2022-01-15 至 2024-12-31
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=2140057&HistoricalAwards=false
关键词：
Collaborative Research AF Small Unified

项目摘要

Data science and machine learning have transformed modern science, engineering, and business. One of the pillars of modern-day machine-learning technology is mathematical optimization, which is the methodology that drives the process of learning from available and/or real-time generated data. Unfortunately, however, despite the successes of certain optimization techniques, large-scale learning remains extremely expensive in terms of time and energy, which puts the ability to train machines to perform certain fundamental tasks exclusively in the hands of those with access to extreme-scale supercomputing facilities. A significant deficiency of many contemporary techniques is that they "launch" an algorithm with a prescribed "trajectory," despite the fact that the actual trajectory that the algorithm will follow depends on unknown factors. Contemporary optimization techniques for machine learning essentially account for this by "tuning" algorithmic parameters, which means that the target is typically only hit after numerous expensive misses. Another significant deficiency of contemporary techniques is the restrictive set of assumptions often made about the optimization being performed, which typically includes the assumption that the machine-learning model is being trained with uncorrupted data. Modern real-world applications are far more complex.This project will explore the design and analysis of adaptive ("self-tuning") optimization techniques for machine learning and related topics. One goal is to produce adaptive algorithms with rigorous guarantees that can avoid the extreme amounts of wasteful computation that are required by contemporary algorithms for parameter tuning. Another goal is to extend the use of these algorithms to settings with imperfect data/information, which may be due to biased function information, corrupted data, or novel techniques for approximating the objective. Finally, many applications ultimately require the learning process or model to satisfy some explicit or implicit constraints. Optimization methods for such machine-learning applications are still in their infancy, largely due to their more complicated nature and further dependence on algorithmic parameters. This project aims to design a unified framework for analyzing adaptive stochastic optimization methods that will offer researchers and practitioners a set of easy-to-use tools for designing next-generation algorithms for cutting-edge applications.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

数据科学和机器学习改变了现代科学、工程和商业。现代机器学习技术的支柱之一是数学优化，它是驱动从可用和/或实时生成的数据中学习的过程的方法。然而不幸的是，尽管某些优化技术取得了成功，大规模学习在时间和精力方面仍然极其昂贵，这使得训练机器执行某些基本任务的能力完全掌握在那些能够访问极端规模的人手中。超级计算设施。许多当代技术的一个显着缺陷是它们“启动”具有规定“轨迹”的算法，尽管算法将遵循的实际轨迹取决于未知因素。现代机器学习优化技术本质上是通过“调整”算法参数来解决这一问题，这意味着通常只有在多次昂贵的失误之后才能达到目标。当代技术的另一个重大缺陷是经常对正在执行的优化做出的限制性假设，其中通常包括机器学习模型正在使用未损坏的数据进行训练的假设。现代现实世界的应用要复杂得多。该项目将探索机器学习和相关主题的自适应（“自调整”）优化技术的设计和分析。目标之一是产生具有严格保证的自适应算法，可以避免当代参数调整算法所需的大量浪费计算。另一个目标是将这些算法的使用扩展到具有不完美数据/信息的设置，这可能是由于有偏差的函数信息、损坏的数据或用于逼近目标的新技术造成的。最后，许多应用最终要求学习过程或模型满足一些显式或隐式的约束。此类机器学习应用的优化方法仍处于起步阶段，这主要是由于其更复杂的性质以及对算法参数的进一步依赖。该项目旨在设计一个用于分析自适应随机优化方法的统一框架，该框架将为研究人员和从业人员提供一套易于使用的工具，用于为尖端应用设计下一代算法。该奖项反映了 NSF 的法定使命，并已通过使用基金会的智力优点和更广泛的影响审查标准进行评估，认为值得支持。