CIF: Small: Online Learning and Optimal Experiment Design with a Budget

CIF：小型：在线学习和预算内的最佳实验设计

• 批准号: 2007036
• 负责人: Kevin Jamieson
• 金额: $ 50.02万
• 依托单位: University of Washington
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2007036&HistoricalAwards=false
• 关键词: CIF; Small; Online; Learning; Optimal

Machine learning is routinely used in science and industry to make inferences about a phenomenon that cannot be observed directly, but can be probed through a series of experiments. For instance, the chief metric when optimizing a chemical reaction may be the yield of the desired output, but many experimental conditions such as pH and ambient temperature may affect the yield. Adaptive experimental design provides a framework to exploit observed measurements of the past to plan measurements in the future in a closed loop. It has been shown to require far fewer overall measurements to achieve the same inference goals compared to any fixed plan chosen in advance. However, a limitation is the implicit assumption that every possible measurement is available at all times. In practice this is rarely true - for example chemical reagents can run out and restrict the possible experiments. This forces a tradeoff on practitioners: if only a subset of measurements are possible at the current time and you have a fixed budget of experiments, is it worth it to take one of the available experiments, or abstain in the hope of better opportunities in the future? The focus of this research is to formalize such questions and develop a framework for addressing online adaptive experimental design in the sequential setting of unpredictable measurement availability. The project also includes a plan to vertically integrate robust data collection techniques across the university touching all levels and disciplines, as well as outreach that starts with K-12 students and extends to the community at large.This project amalgamates insights from adaptive experimental design, multi-armed bandits, and online algorithms. Current adaptive experimental design methods, for instance in stochastic optimization and best-arm identification, assume access to a fixed batch of experiments to choose from at each time, and explicitly plan to evolve the allocation of measurements over this batch using optimal design techniques such as G-optimal design. However, if the measurement set is changing at each time, potentially adversarially, such planning is extremely difficult. Motivated by progress in specific cases that leverage advances in convex optimization, the project seeks to provide a general framework for experimental design including optimization and multiple testing in online settings.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

机器学习通常用于科学和工业中，以对无法直接观察但可以通过一系列实验进行探究的现象进行推断。例如，优化化学反应时的主要指标可能是所需输出的产率，但许多实验条件（例如 pH 值和环境温度）可能会影响产率。自适应实验设计提供了一个框架，可以利用过去观察到的测量结果来在闭环中规划未来的测量结果。事实证明，与提前选择的任何固定计划相比，实现相同推理目标所需的总体测量要少得多。然而，一个限制是隐含的假设，即每种可能的测量在任何时候都是可用的。在实践中，这种情况很少发生——例如化学试剂可能会耗尽并限制可能的实验。这迫使从业者做出权衡：如果当前只能进行测量的子集，并且您有固定的实验预算，那么是否值得进行其中一个可用的实验，或者放弃以期获得更好的机会未来？这项研究的重点是将这些问题形式化，并开发一个框架，用于在不可预测的测量可用性的顺序设置中解决在线自适应实验设计。该项目还包括一项在整个大学垂直整合涉及各个级别和学科的强大数据收集技术的计划，以及从 K-12 学生开始并扩展到整个社区的外展活动。该项目融合了来自适应性实验设计、多臂老虎机和在线算法。当前的自适应实验设计方法，例如随机优化和最佳臂识别，假设每次都可以访问固定批次的实验以供选择，并明确计划使用最优设计技术来改进该批次的测量分配，例如G 最优设计。然而，如果测量集每次都在变化，可能会产生不利影响，那么这样的规划就极其困难。受利用凸优化进步的特定案例取得进展的推动，该项目旨在为实验设计提供一个通用框架，包括在线环境中的优化和多重测试。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准。

项目成果

Experimental Design for Regret Minimization in Linear Bandits

• 发表时间: 2020-11
• 作者: Andrew Wagenmaker;Julian Katz-Samuels;Kevin G. Jamieson

Best Arm Identification with Safety Constraints

具有安全约束的最佳手臂识别

• 发表时间: 2021
• 期刊: Proceedings of Machine Learning Research
• 作者: Wang, Zhenlin;Wagenmaker, Andrew;Jamieson, Kevin
• 通讯作者: Jamieson, Kevin

Stochastic Contextual Bandits with Long Horizon Rewards

具有长期奖励的随机上下文强盗

• 发表时间: 2023
• 期刊: Proceedings of the AAAI Conference on Artificial Intelligence
• 作者: Qin, Yuzhen;Li, Yingcong;Pasqualetti, Fabio;Fazel, Maryam;Oymak, Samet
• 通讯作者: Oymak, Samet

High-Dimensional Experimental Design and Kernel Bandits

• 发表时间: 2021-05
• 作者: Romain Camilleri;Julian Katz-Samuels;Kevin G. Jamieson

Near-Optimal Randomized Exploration for Tabular Markov Decision Processes

• 发表时间: 2021-02
• 作者: Zhihan Xiong;Ruoqi Shen;Qiwen Cui;Maryam Fazel;S. Du