Building Reinforcement Learning and Normative Models in the Cloud

在云中构建强化学习和规范模型

• 批准号: 10825877
• 负责人: POORNIMA KUMAR
• 金额: $ 24.58万
• 依托单位: MCLEAN HOSPITAL
• 依托单位国家: 美国
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-12-01 至 2024-07-31
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/10825877
• 关键词: Acceleration; Address; Adopted; Award; Bayesian Method; Behavior; Behavioral; Clinic; Cloud Computing; Communities; Complex; Computer Models; Computer Systems; Conflict (Psychology); Cost Savings; Data; Data Analyses; Data Collection; Data Reporting; Data Storage and Retrieval; Decision Making; Estimation Techniques; Fingerprint; Funding; Future; Goals; Growth; Heterogeneity; Hospitals; Impairment; Individual; Learning; Mathematics; Mental disorders; Modeling; Outcome; Parameter Estimation; Parents; Participant; Patient Self-Report; Patients; Process; Psychiatry; Psychological reinforcement; Reproducibility; Research; Resource Sharing; Resources; Running; Sample Size; Sampling; Services; Stratification; Techniques; Testing; Time; aged; clinical phenotype; cluster computing; computer framework; computing resources; cost; cost effective; data exchange; data integration; discounting; falls; improved; machine learning algorithm; novel; online resource; parent grant; parent project; precision medicine; prototype; recruit; tool; virtual

The parent proposal aims to address a critical need in the field of precision psychiatry by identifying a complex behavior, such as Reinforcement Learning based Decision Making (RLDM), which is impaired across various psychiatric disorders and adopting a computational framework to explain heterogeneity at an individual level. By building normative models of RLDM constructs and charting heterogeneity at the individual level, the proposal aims to advance precision medicine. The main goals of this proposal are to (1) parse RLDM sub- processes into mathematically-defined parameters in a large sample using a diverse set of tasks; (2) assess test-retest reliability of these parameters; and finally (3) build normative models of the parameters and chart the heterogeneity at the level of the individual. We will reach these goals by collecting behavioral data from a diverse set of tasks in a large community sample (n=1000) and 500 of these participants will complete the tasks a second time within two weeks to enable us to assess test-retest reliability of the computationally-derived RLDM parameters. The framework that was proposed in the parent R21 involved deploying 6 RLDM tasks online and collecting data using one of the cloud/cloud-like services such as AWS, Pavlovia or testmybrain. We were then planning on downloading all the behavioral data and running our RLDM and normative models in our local compute cluster, due to limited funds available in the parent R21 to use cloud computing. In this proposal, we aim to conduct the entirety of our project on the cloud with the funds provided by this supplement. The entire parent project could benefit tremendously from having access to the cloud resources – from online tasks deployment, data collection and automated large scale computationally intensive data analyses. Running RLDM models and creating normative charts are computationally intensive and require significant resources. Our plan was to collect data from six tasks, run three to five RLDM models on each task, estimate RLDM parameters and develop normative models of the eight most stable RLDM parameters in 1000 participants. However, with the use of affordable cloud computing through this supplement, we will be able to not only vastly reduce computational time (which would be very slow on our computing cluster that is a shared resource across the Hospital), but this will also give us an opportunity to explore complex RLDM models and test novel estimation techniques. Additionally, by reducing the burden on local compute clusters and costs (budgeted in our parent grant), we might be able to increase our originally proposed sample size, thereby enhancing the robustness of the normative models with data from a larger sample size. With the entire project on the cloud, there will be seamless integration from data collection to data analyses and statistical interpretation, which will improve the overall efficiency of the project.

母提案旨在通过确定一个解决精准精神病学领域的关键需求 复杂的行为，例如基于强化学习的决策（RLDM），它在各个方面都受到损害 各种精神疾病并采用计算框架来解释个体的异质性 通过建立 RLDM 结构的规范模型并绘制个体层面的异质性图表， 该提案旨在推进精准医学。该提案的主要目标是（1）解析 RLDM 子模块。 (2) 使用一组不同的任务将过程评估为大样本中数学定义的参数； 重测这些参数的可靠性；最后（3）建立参数的规范模型并绘制图表 我们将通过从不同的人那里收集行为数据来实现这些目标。 大型社区样本 (n=1000) 中的一组任务，其中 500 名参与者将在一秒钟内完成这些任务 两周内的时间使我们能够评估计算得出的 RLDM 的重测可靠性 父 R21 中提出的框架涉及在线部署 6 个 RLDM 任务。 我们当时使用一种云/类云服务（例如 AWS、Pavlovia 或 testmybrain）收集数据。 计划下载所有行为数据并在本地运行我们的 RLDM 和规范模型 计算集群，由于母公司 R21 中可用的资金有限，因此我们在本提案中使用了云计算。 旨在利用本补充提供的资金在云上执行我们的整个项目。 整个父项目可以通过访问云资源（通过在线任务）获得巨大收益 部署、数据收集和自动化大规模计算密集型数据分析。 模型和创建规范图表需要大量计算，并且需要大量资源。 是从六个任务收集数据，在每个任务上运行三到五个 RLDM 模型，估计 RLDM 参数并 然而，在 1000 名参与者中开发 8 个最稳定的 RLDM 参数的规范模型。 通过这种补充使用负担得起的云计算，我们不仅能够大大减少 计算时间（对于我们的计算集群来说，这会非常慢，因为计算集群是跨网络的共享资源） Hospital），但这也让我们有机会探索复杂的 RLDM 模型并测试新颖的估计 此外，通过减少本地计算集群的负担和成本（在我们的父级中进行预算） grant），我们也许能够增加我们最初提出的样本量，从而增强 随着整个项目都在云端，将会有来自更大样本量的数据的规范模型。 从数据收集到数据分析和统计解释的无缝集成，这将提高 项目的整体效率。