CIF: Small: Online Learning and Optimal Experiment Design with a Budget

CIF：小型：在线学习和预算内的最佳实验设计

• 批准号: 2007036
• 负责人: Kevin Jamieson
• 金额: $ 50.02万
• 依托单位: University of Washington
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2007036&HistoricalAwards=false
• 关键词: CIF; Small; Online; Learning; Optimal

Machine learning is routinely used in science and industry to make inferences about a phenomenon that cannot be observed directly, but can be probed through a series of experiments. For instance, the chief metric when optimizing a chemical reaction may be the yield of the desired output, but many experimental conditions such as pH and ambient temperature may affect the yield. Adaptive experimental design provides a framework to exploit observed measurements of the past to plan measurements in the future in a closed loop. It has been shown to require far fewer overall measurements to achieve the same inference goals compared to any fixed plan chosen in advance. However, a limitation is the implicit assumption that every possible measurement is available at all times. In practice this is rarely true - for example chemical reagents can run out and restrict the possible experiments. This forces a tradeoff on practitioners: if only a subset of measurements are possible at the current time and you have a fixed budget of experiments, is it worth it to take one of the available experiments, or abstain in the hope of better opportunities in the future? The focus of this research is to formalize such questions and develop a framework for addressing online adaptive experimental design in the sequential setting of unpredictable measurement availability. The project also includes a plan to vertically integrate robust data collection techniques across the university touching all levels and disciplines, as well as outreach that starts with K-12 students and extends to the community at large.This project amalgamates insights from adaptive experimental design, multi-armed bandits, and online algorithms. Current adaptive experimental design methods, for instance in stochastic optimization and best-arm identification, assume access to a fixed batch of experiments to choose from at each time, and explicitly plan to evolve the allocation of measurements over this batch using optimal design techniques such as G-optimal design. However, if the measurement set is changing at each time, potentially adversarially, such planning is extremely difficult. Motivated by progress in specific cases that leverage advances in convex optimization, the project seeks to provide a general framework for experimental design including optimization and multiple testing in online settings.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

机器学习通常用于科学和工业中，以推断无法直接观察到的现象，但可以通过一系列实验来探测。例如，优化化学反应时的主要指标可能是所需输出的产量，但是许多实验条件（例如pH和环境温度）可能会影响产量。自适应实验设计提供了一个框架，以利用观察到的过去的测量值，以在将来在封闭环中计划测量。与提前选择的任何固定计划相比，已经显示出更少的总体测量要达到相同的推断目标。但是，限制是隐含的假设，即每个可能的测量始终可用。在实践中，这很少是正确的 - 例如，化学试剂可能用完并限制可能的实验。这迫使对从业者进行权衡：如果目前只有一部分测量值，并且您有固定的实验预算，那么接受一项可用的实验还是戒除，希望将来有更好的机会吗？这项研究的重点是正式化此类问题并开发一个框架，以在无法预测的测量可用性的顺序环境中解决在线自适应实验设计。该项目还包括一项计划，以垂直整合整个大学的强大数据收集技术，以触摸各个层次和学科，以及从K-12学生开始，并扩展到整个社区。该项目Amalgamates从自适应实验设计，多臂武装bastits和在线算法中介绍。当前的自适应实验设计方法，例如，在随机优化和最佳臂识别中，都会访问每次固定的实验供您选择的固定实验，并明确计划使用最佳设计技术（例如G-optimal Design）在此批次上分配测量结果。但是，如果测量集在每个时间都在改变，那么在对手方面可能会非常困难。在利用凸优化方面进步的特定情况下的进展，该项目旨在为实验设计提供一个一般框架，包括优化和在线环境中的多次测试。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的知识分子优点和更广泛的审查标准来通过评估来通过评估来提供支持的。

项目成果

Experimental Design for Regret Minimization in Linear Bandits

• 发表时间: 2020-11
• 作者: Andrew Wagenmaker;Julian Katz-Samuels;Kevin G. Jamieson

Best Arm Identification with Safety Constraints

具有安全约束的最佳手臂识别

• 发表时间: 2021
• 期刊: Proceedings of Machine Learning Research
• 作者: Wang, Zhenlin;Wagenmaker, Andrew;Jamieson, Kevin
• 通讯作者: Jamieson, Kevin

Stochastic Contextual Bandits with Long Horizon Rewards

具有长期奖励的随机上下文强盗

• 发表时间: 2023
• 期刊: Proceedings of the AAAI Conference on Artificial Intelligence
• 作者: Qin, Yuzhen;Li, Yingcong;Pasqualetti, Fabio;Fazel, Maryam;Oymak, Samet
• 通讯作者: Oymak, Samet

High-Dimensional Experimental Design and Kernel Bandits

• 发表时间: 2021-05
• 作者: Romain Camilleri;Julian Katz-Samuels;Kevin G. Jamieson

Near-Optimal Randomized Exploration for Tabular Markov Decision Processes

• 发表时间: 2021-02
• 作者: Zhihan Xiong;Ruoqi Shen;Qiwen Cui;Maryam Fazel;S. Du