Collaborative Research: High-Dimensional Decision Making and Inference with Applications for Personalized Medicine

合作研究：高维决策和推理及其在个性化医疗中的应用

• 批准号: 2015568
• 负责人: Zhaoran Wang
• 金额: $ 10万
• 依托单位: Northwestern University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-06-15 至 2023-05-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2015568&HistoricalAwards=false
• 关键词: Collaborative; Research; Dimensional; Decision; Making

With the advent of data collection and storage technology, researchers can obtain large-scale and high-dimensional datasets at a low price. Such datasets offer exciting opportunities to make better decisions and reveal new discoveries to improve decision making in various applications, and meanwhile, also raise statistical challenges. Over the past decades, regularization methods such as Lasso, SCAD, and MCP have been proposed to conduct model estimation in the presence of high dimensional covariates. Various numerical algorithms have been developed for these methods, and their theoretical properties are well studied. However, questions of how to efficiently and effectively utilize high-dimensional data to make optimal decisions and conduct inference are relatively less studied, although such problems are of vital practical importance. This project will develop new methods and theories for making optimal decisions and conducting valid inference under high-dimensional settings. The methods have wide applications, for instance, in personalized medicine where the goal is to determine the optimal treatments for a patient based on predictor information, including several thousand genetic markers. The principal investigators will develop and distribute user-friendly open-source software to practitioners and provide training opportunities to students at different levels. The project has three research aims. The first aim is to study the high-dimensional contextual bandit problem with binary actions, which is an online decision-making problem that finds applications in personalized healthcare and precision medicine. In this problem, the player sequentially chooses one action and observes a reward, where the goal is to maximize the reward. The principal investigators will develop a new algorithm to provide an optimal decision rule, which achieves the minimax optimal regret. The second aim is to study general inference problems that arise from high-dimensional stochastic convex optimization, where the goal is to quantify the uncertainties of the optimal objective value. The third goal is to consider the general stochastic linear bandit problem with a finite and random action space. The principal investigators will develop a new algorithm by using a best-subset-selection type estimator, and the approach achieves a "dimension-free" regret and meets existing lower-bound under the low-dimensional setting.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

随着数据采集和存储技术的出现，研究人员可以以低廉的价格获得大规模、高维的数据集。此类数据集提供了令人兴奋的机会来做出更好的决策并揭示新发现以改进各种应用中的决策，同时也提出了统计挑战。在过去的几十年里，Lasso、SCAD 和 MCP 等正则化方法被提出来在存在高维协变量的情况下进行模型估计。人们已经为这些方法开发了各种数值算法，并且对它们的理论特性进行了深入研究。然而，如何高效、有效地利用高维数据进行最优决策和推理的问题却相对较少，尽管此类问题具有至关重要的实际意义。该项目将开发新的方法和理论，以在高维设置下做出最优决策并进行有效推理。这些方法具有广泛的应用，例如，在个性化医疗中，其目标是根据预测信息（包括数千个遗传标记）确定患者的最佳治疗方法。主要研究人员将开发并向从业者分发用户友好的开源软件，并为不同级别的学生提供培训机会。该项目有三个研究目标。第一个目标是研究具有二元动作的高维上下文强盗问题，这是一个在线决策问题，可在个性化医疗和精准医疗中找到应用。在这个问题中，玩家依次选择一个动作并观察奖励，其目标是最大化奖励。主要研究人员将开发一种新算法来提供最优决策规则，从而实现最小最大最优后悔。第二个目标是研究高维随机凸优化产生的一般推理问题，其目标是量化最优目标值的不确定性。第三个目标是考虑具有有限和随机动作空间的一般随机线性老虎机问题。主要研究人员将通过使用最佳子集选择类型估计器开发一种新算法，该方法实现了“无维”遗憾并满足低维设置下的现有下限。该奖项反映了 NSF 的法定使命和通过使用基金会的智力价值和更广泛的影响审查标准进行评估，该项目被认为值得支持。

项目成果

Online Bootstrap Inference For Policy Evaluation In Reinforcement Learning

强化学习中策略评估的在线引导推理

• DOI: 10.1080/01621459.2022.2096620
• 发表时间: 2021-08-08
• 期刊: Journal of the American Statistical Association
• 影响因子: 3.7
• 作者: Pratik Ramprasad;Yuantong Li;Zhuoran Yang;Zhaoran Wang;W. Sun;Guang Cheng
• 通讯作者: Guang Cheng