Collaborative Research: AF: Small: A Unified Framework for Analyzing Adaptive Stochastic Optimization Methods Based on Probabilistic Oracles

合作研究：AF：Small：基于概率预言的自适应随机优化方法分析统一框架

基本信息

批准号：
2139735
负责人：
Frank Curtis
金额：
$ 25万
依托单位：
Lehigh University
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2022
资助国家：
美国
起止时间：
2022-01-15 至 2024-12-31
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=2139735&HistoricalAwards=false
关键词：
Collaborative Research AF Small Unified

项目摘要

Data science and machine learning have transformed modern science, engineering, and business. One of the pillars of modern-day machine-learning technology is mathematical optimization, which is the methodology that drives the process of learning from available and/or real-time generated data. Unfortunately, however, despite the successes of certain optimization techniques, large-scale learning remains extremely expensive in terms of time and energy, which puts the ability to train machines to perform certain fundamental tasks exclusively in the hands of those with access to extreme-scale supercomputing facilities. A significant deficiency of many contemporary techniques is that they "launch" an algorithm with a prescribed "trajectory," despite the fact that the actual trajectory that the algorithm will follow depends on unknown factors. Contemporary optimization techniques for machine learning essentially account for this by "tuning" algorithmic parameters, which means that the target is typically only hit after numerous expensive misses. Another significant deficiency of contemporary techniques is the restrictive set of assumptions often made about the optimization being performed, which typically includes the assumption that the machine-learning model is being trained with uncorrupted data. Modern real-world applications are far more complex.This project will explore the design and analysis of adaptive ("self-tuning") optimization techniques for machine learning and related topics. One goal is to produce adaptive algorithms with rigorous guarantees that can avoid the extreme amounts of wasteful computation that are required by contemporary algorithms for parameter tuning. Another goal is to extend the use of these algorithms to settings with imperfect data/information, which may be due to biased function information, corrupted data, or novel techniques for approximating the objective. Finally, many applications ultimately require the learning process or model to satisfy some explicit or implicit constraints. Optimization methods for such machine-learning applications are still in their infancy, largely due to their more complicated nature and further dependence on algorithmic parameters. This project aims to design a unified framework for analyzing adaptive stochastic optimization methods that will offer researchers and practitioners a set of easy-to-use tools for designing next-generation algorithms for cutting-edge applications.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

数据科学和机器学习改变了现代科学，工程和业务。现代机器学习技术的支柱之一是数学优化，这是驱动从可用和/或实时生成的数据中学习过程的方法。但是，不幸的是，尽管某些优化技术取得了成功，但在时间和精力方面，大规模学习仍然非常昂贵，这使得训练机器以专门掌握具有极高规模超级分配设施的人的手中执行某些基本任务。许多当代技术的一个重大缺陷是，尽管它们的实际轨迹将遵循的实际轨迹取决于未知因素，但它们“启动”具有规定“轨迹”的算法。当代机器学习的现代优化技术基本上通过“调整”算法参数来解释这一点，这意味着该目标通常仅在经过许多昂贵的失误之后才能击中。当代技术的另一个重要缺陷是通常对要进行优化做出的限制性假设集，这通常包括假设机器学习模型正在使用未腐败的数据进行训练。现代现实世界的应用程序更为复杂。该项目将探讨用于机器学习和相关主题的自适应（“自调”）优化技术的设计和分析。一个目标是生产具有严格保证的自适应算法，以避免当代算法进行参数调整所需的极端浪费计算。另一个目标是将这些算法的使用扩展到具有不完美的数据/信息的设置，这可能是由于功能信息，损坏的数据或新颖的技术来近似目标。最后，许多应用程序最终需要学习过程或模型来满足某些明确或隐式约束。此类机器学习应用程序的优化方法仍处于起步阶段，这主要是由于它们更复杂的性质和进一步依赖算法参数。该项目旨在设计一个统一的框架，用于分析适应性的随机优化方法，该方法将为研究人员和从业人员提供一套易于使用的工具，用于设计用于尖端应用的下一代算法。该奖项反映了NSF的法规任务，并被认为是通过基金会的知识优点和广泛的范围来评估的。