Engaging Multidisciplinary Health System Stakeholders to Create a Process for Implementing Machine-Learning Enabled Clinical Decision Support

让多学科卫生系统利益相关者参与创建实施机器学习支持的临床决策支持的流程

• 批准号: 10656387
• 负责人: Benjamin Alan Goldstein
• 金额: $ 21.01万
• 依托单位: DUKE UNIVERSITY
• 依托单位国家: 美国
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-07-01 至 2024-06-30
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/10656387
• 关键词: Adopted; Adoption; Affect; Behavior; Biological Models; Clinical; Community Hospitals; Consensus; Data; Data Science; Data Scientist; Data Set; Decision Making; Decision Support Model; Feedback; Frequencies; Future; Goals; Health system; Healthcare Systems; Institution; Intervention; Knowledge; Learning; Machine Learning; Maps; Measures; Medical Device; Medicine; Methodology; Methods; Modeling; Monitor; Pathway interactions; Patients; Performance; Phase; Process; Process Measure; Proliferating; Provider; Public Health; Regulation; Research; Risk; Safety; Series; Service setting; System; Techniques; Technology; Time; Trust; United States National Library of Medicine; Work; bioinformatics tool; care delivery; clinical care; clinical decision support; clinical decision-making; complex data; cost estimate; dynamic system; evidence base; implementation barriers; implementation evaluation; implementation outcomes; implementation science; improved; innovation; insight; machine learning algorithm; machine learning model; model building; models and simulation; multidisciplinary; prospective; research clinical testing; response; support tools; theories; tool; uptake; usability; Adopted; Adoption; Affect; Behavior; Biological Models; Clinical; Community Hospitals; Consensus; Data; Data Science; Data Scientist; Data Set; Decision Making; Decision Support Model; Feedback; Frequencies; Future; Goals; Health system; Healthcare Systems; Institution; Intervention; Knowledge; Learning; Machine Learning; Maps; Measures; Medical Device; Medicine; Methodology; Methods; Modeling; Monitor; Pathway interactions; Patients; Performance; Phase; Process; Process Measure; Proliferating; Provider; Public Health; Regulation; Research; Risk; Safety; Series; Service setting; System; Techniques; Technology; Time; Trust; United States National Library of Medicine; Work; bioinformatics tool; care delivery; clinical care; clinical decision support; clinical decision-making; complex data; cost estimate; dynamic system; evidence base; implementation barriers; implementation evaluation; implementation outcomes; implementation science; improved; innovation; insight; machine learning algorithm; machine learning model; model building; models and simulation; multidisciplinary; prospective; research clinical testing; response; support tools; theories; tool; uptake; usability

PROJECT SUMMARY/ABSTRACT The proliferation of “black box” Machine Learning (ML) models for Clinical Decision Support (CDS) has raised concerns regarding CDS interpretability, actionability and overall usability, rendering a critical need for a clear process that engages various stakeholders including both developers and users in implementation planning. Our long-term goal is to formalize a process to guide health systems in planning, monitoring and evaluating CDS implementation. The overall objective for this R21 is to develop and evaluate a generalizable strategy to bring multidisciplinary stakeholders together during the CDS exploration phase to identify facilitators and barriers to implementation in their contexts. In doing so, we will use Participatory System Dynamics (PSD) modeling as a multi-component strategy to evaluate and plan implementation with stakeholders during the exploration phase of implementation, when decision-making occurs, in a way where ML-enabled CDS can be sustained over time. As such, we will focus on the upstream implementation outcomes of acceptability, appropriateness, and feasibility of ML-enabled CDS. The rationale for this project is that a process that engages diverse stakeholders in implementation planning early on will clarify commitment to implementation and potential for adoption by revealing acceptability, feasibility, and appropriateness. For this project we will focus on one particular set of ML-enabled CDS: Early Warning Scores (EWSs), used to identify decompensating patients. We plan to accomplish our overall objective by pursuing two specific aims: 1. Engage multidisciplinary stakeholders involved in EWS implementation (users, developers, implementers, owners) from two systematically varying adoption contexts to co-define common barriers and facilitators to key implementation outcomes of CDS acceptability, appropriateness, and feasibility using group model building scripts from the field of system dynamics and 2. Evaluate the PSD process by measuring change in commitment to adopt CDS (using measures of acceptability, appropriateness, and feasibility), eliciting feedback, and estimating intervention effort. We will obtain data via a series of group modeling sessions from stakeholders who have used CDS in different contexts, where alerts vary by target user, time, and frequency among other factors. We will employ well-defined scripts from the field of System Dynamics modeling to facilitate group discussion toward developing a shared theory about the problem of ML-enabled CDS response (Aim 1). Because implementation of any strategy requires adaptation, we will evaluate the PSD process (Aim 2) to refine and prepare for use elsewhere. This contribution is significant because EWSs are widely used across both academic and community hospitals. This contribution is innovative by using group modeling techniques for the problem of ML-enabled CDS implementation, creating both methodological and substantive findings. A future R01 will prospectively assess benefits of using this process in multiple use case settings while continuing to build out the dynamic systems model of factors for downstream adoption.

项目摘要/摘要 临床决策支持（CD）的“黑匣子”机器学习（ML）模型的扩散已提高 对CD的可解释性，可行性和整体可用性的担忧，这使人们对清晰的需求非常重要 与包括开发人员和用户在实施计划中的各种利益相关者相关的过程。 我们的长期目标是为指导卫生系统进行计划，监视和评估的过程形式化 CDS实施。 R21的总体目标是制定和评估可推广的策略 在CDS勘探阶段将多学科利益相关者召集在一起 在其背景下实施的促进者和障碍。这样，我们将使用参与系统 动态（PSD）建模是一种多组分策略，用于评估和计划实施 在实施探索阶段，利益相关者在决策时以某种方式 启用ML的CD可以随着时间的流逝而维持。因此，我们将重点关注上游实施 启用ML的CD的可接受性，适当性和可行性的结果。这个项目的理由是 该过程与潜在的利益相关者尽早参与实施计划的过程将澄清承诺 通过揭示可接受性，可行性和适当性来实施和采用潜力。为了 该项目我们将重点介绍一组启用ML的CD：预警分数（EWSS），曾经曾经 识别失代偿患者。我们计划通过追求两个具体目标来实现我们的整体目标：1。 参与参与EWS实施的多学科利益相关者（用户，开发人员，实施者， 所有者）从两个系统变化的采用环境到共同定义的共同障碍和促进者到关键 使用小组模型建设的CD可接受性，适当性和可行性的实施结果 来自系统动力学领域和2的脚本。通过测量更改来评估PSD过程 承诺采用CD（使用可接受性，适当性和可行性的度量），引起 反馈和估计干预工作。我们将通过一系列的小组建模会话获得数据 在不同情况下使用CD的利益相关者，在目标用户，时间和频率的情况下，警报会有所不同 除其他因素。我们将使用系统动态建模领域定义明确的脚本到 促进小组讨论，以发展有关启用ML的CD响应问题的共同理论 （目标1）。因为任何策略的实施都需要适应，所以我们将评估PSD流程（AIM 2）精炼并准备在其他地方使用。这种贡献很重要，因为EWS被广泛使用 在学术和社区医院中。通过使用小组建模，这种贡献是创新的 启用ML的CD实施问题的技术，创建方法论和实质性 发现。未来的R01可能会评估在多个用例设置中使用此过程的好处 在继续建立下游采用因素的动态系统模型的同时。