NSF-BSF: AF: Small: Parameter-Free Stochastic Optimization via Trajectory Cues

NSF-BSF：AF：小：通过轨迹线索进行无参数随机优化

基本信息

批准号：
2239527
负责人：
Oliver Hinder
金额：
$ 45.27万
依托单位：
University of Pittsburgh
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2023
资助国家：
美国
起止时间：
2023-05-01 至 2026-04-30
项目状态：
未结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=2239527&HistoricalAwards=false
关键词：
NSF BSF AF Small Parameter

项目摘要

Machine learning has been extremely successful at enabling computers to learn from data to perform image annotation, speech recognition, and language translation. Despite this success, many potential applications remain untapped. One major barrier to more widespread adoption of machine learning is that machine learning methods must be manually tuned to each new learning task. This tuning requires high levels of both computational resources and user skill, which excludes many potential machine learning users. Furthermore, even leading machine learning practitioners make costly tuning mistakes. This project aims to make the tuning process simpler and more computationally efficient.This project focuses on eliminating parameter tuning from the stochastic optimization algorithms that power machine learning. Toward this goal, two novel strategies will be employed. First, instead of using online-to-batch conversion, a standard technique in stochastic optimization, this project will develop algorithms for noiseless optimization and then lift them to the stochastic setting via powerful time-uniform concentration of measure results. Second, the project will develop algorithms based on trajectory cues: quantities directly observable from the optimization trajectory (such as the distance between iterates and the initial point) that serve as stand-ins for unknown problem parameters. These two new strategies will enable this project to tackle outstanding theoretical problems in parameter-free optimization and design practical algorithms that reduce the burden of parameter tuning. The project will also examine parameter-free methods for particular problem classes such as deep learning. All software resulting from this research will be open source. Algorithms from this project will be integrated into optimization and machine learning curricula at both the undergraduate and graduate level.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

机器学习在使计算机能够从数据中学习以执行图像注释、语音识别和语言翻译方面取得了巨大成功。尽管取得了如此成功，但许多潜在的应用仍未开发。更广泛地采用机器学习的一个主要障碍是机器学习方法必须针对每个新的学习任务进行手动调整。这种调整需要高水平的计算资源和用户技能，这排除了许多潜在的机器学习用户。此外，即使是领先的机器学习从业者也会犯下代价高昂的调整错误。该项目旨在使调整过程更简单、计算效率更高。该项目的重点是消除驱动机器学习的随机优化算法中的参数调整。为了实现这一目标，将采用两种新颖的策略。首先，该项目将开发无噪声优化算法，然后通过强大的时间均匀集中测量结果将其提升到随机设置，而不是使用在线批量转换（随机优化中的标准技术）。其次，该项目将开发基于轨迹线索的算法：从优化轨迹中可直接观察到的量（例如迭代与初始点之间的距离），作为未知问题参数的替代。这两种新策略将使该项目能够解决无参数优化中突出的理论问题，并设计减轻参数调整负担的实用算法。该项目还将研究针对特定问题类别（例如深度学习）的无参数方法。这项研究产生的所有软件都将是开源的。该项目的算法将被纳入本科生和研究生的优化和机器学习课程中。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

期刊论文数量（3）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

DoG is SGD’s Best Friend: A Parameter-Free Dynamic Step Size Schedule

DoG 是 SGD 最好的朋友：无参数动态步长计划

DOI：
发表时间：
2023-07
期刊：
Proceedings of the 40th International Conference on Machine Learning
影响因子：
0
作者：
Maor Ivgi; Oliver Hinder
通讯作者：
Oliver Hinder

Datasets and Benchmarks for Nanophotonic Structure and Parametric Design Simulations

纳米光子结构和参数化设计模拟的数据集和基准

DOI：
10.48550/arxiv.2310.19053
发表时间：
2023-10-29
期刊：
ArXiv
影响因子：
0
作者：
Jungtaek Kim;Mingxuan Li;Oliver Hinder;Paul W. Leu
通讯作者：
Paul W. Leu

Multi-BOWS: multi-fidelity multi-objective Bayesian optimization with warm starts for nanophotonic structure design

Multi-BOWS：纳米光子结构设计的多保真度多目标贝叶斯优化和热启动

DOI：
10.1039/d3dd00177f
发表时间：
2024-01
期刊：
Digital Discovery
影响因子：
0
作者：
Kim, Jungtaek;Li, Mingxuan;Li, Yirong;Gómez, Andrés;Hinder, Oliver;Leu, Paul W.
通讯作者：
Leu, Paul W.