Benchmarking Hospital Quality: Template Matching versus Conventional Regression Approaches

医院质量基准测试：模板匹配与传统回归方法

• 批准号: 9679239
• 负责人: Hallie Christine Prescott
• 金额: --
• 依托单位: VETERANS HEALTH ADMINISTRATION
• 依托单位国家: 美国
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-03-01 至 2022-02-28
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/9679239
• 关键词: Academy; Accident and Emergency department; Accountability; Acute Kidney Failure; Algorithms; Ambulatory Care Facilities; Area; Benchmarking; Caring; Case Mixes; Charge; Clinical; Community Care Networks; Complex; Consultations; Creatinine; Data; Discipline of Nursing; Evaluation; Foundations; Future; Goals; Healthcare Systems; Heart; Heart failure; Hospitalization; Hospitals; Individual; Infrastructure; Inpatients; Interview; Investments; Learning; Measurement; Measures; Medical; Medicare claim; Medicine; Methodology; Methods; Modeling; Nursing Homes; Operative Surgical Procedures; Outcome; Outcome Measure; Patient-Focused Outcomes; Patients; Performance; Private Sector; Quality of Care; Recording of previous events; Reporting; Research; Residual state; Resources; Risk; Risk Adjustment; Sampling; Science; Severity of illness; Specific qualifier value; Standardization; Structure; Surveys; System; Testing; Trust; Veterans; Work; acute care; base; follow-up; hospital performance; hospital readmission; improved; innovation; interest; mortality; novel strategies; patient population; profiles in patients; programs; simulation

Background: Identifying and remediating low-quality care is at the heart of systematic quality improvement, particularly in VA. However, cross-hospital comparisons to identify under-performing hospitals are limited by differences in patient case-mix and illness severity. Clinicians consider the current approach to benchmarking with conventional regression to be unfair, unclear, and unhelpful. While substantial resources are devoted to hospital benchmarking, the current return on this investment is limited because clinicians do not understand or trust the methods. The National Academy of Medicine has recently called for investing into the science of performance measurement and for increasing its transparency and validity. Template matching has been proposed as an alternative methodological approach to benchmarking that is fair, clear, and helpful. However, this new approach has never been tested outside of limited research settings. Specific Aims: To test the utility of template matching for comparing quality of care across VA's diverse acute care hospitals, this project will assess the feasibility, accuracy, and interpretability of this approach. Specifically, the project will: (A1) Feasibility: Develop and optimize two template matching approaches for comparing 30-day mortality across VA hospitals. (A2) Accuracy: Compare the ability of template matching versus conventional regression to correctly identify under-performing hospitals. (A3) Interpretability: Compare the interpretability and credibility of hospital performance data generated from template matching versus conventional regression models with clinical leaders. Anticipated Impact: The overarching goal of this proposal is to improve the care of hospitalized Veterans. This proposal will apply promising, new template matching approaches for benchmarking to the diverse VA healthcare system and complete a multi-faceted evaluation of these approaches. We will consider not only the feasibility and methodological rigor of template matching, but also the interpretability, credibility and accountability of the data. We expect that template matching will be feasible for benchmarking VA acute care hospitals, that it will identify under-performing hospitals at least as well as current benchmarking with conventional regression, and that it will be more interpretable and credible to VA Chiefs of Medicine than current performance reports. We anticipate many future expansions to this work, including the use of template matching for VA to private sector comparisons. Unique Features and Innovation: This proposal is innovative because it will simultaneously (1) advance the statistical state-of-the-art in hospital benchmarking, (2) develop the necessary infrastructure in consultation with operational partners to use template matching in VA, and (3) evaluate this new approach not just statistically, but also with the actual users it is supposed to inform. The results are highly relevant to VA, whose performance is always under scrutiny. It is also of great interest outside VA, for the many healthcare systems who similarly struggle to provide meaningful and actionable assessments of individual hospital's performance. Project Methods: Aim 1 will develop and refine the statistical methodology for benchmarking 30-day mortality in VA acute care hospitals using two template matching approaches: (1) a single template and (2) personalized templates for each hospital. Multiple statistical approaches to matching will be considered to achieve the fairest comparisons across hospitals. Aim 2 will use simulation and real patient data to measure when and why template matching and conventional regression approaches may yield discordant assessments of hospitals. Aim 3 will develop the presentation of template matching data, then survey VA Chiefs of Medicine to compare the interpretability and credibility of performance reports using template matching versus conventional regression for hospital benchmarking, with follow-up semi-structured interviews with a subset of Chiefs to further understand template matching performance reports from the perspective of Chiefs of Medicine.

背景：识别和补救低质量护理是系统质量改进的核心， 特别是在弗吉尼亚州。然而，通过跨医院比较来识别表现不佳的医院受到以下因素的限制： 患者病例组合和疾病严重程度的差异。临床医生考虑当前的基准测试方法 传统的回归是不公平的、不明确的且无益的。虽然投入大量资源 医院对标，目前这项投资的回报有限，因为临床医生不了解或不了解 相信方法。美国国家医学院最近呼吁投资于科学 绩效衡量并提高其透明度和有效性。模板匹配已 提出作为基准测试的替代方法，该方法是公平、清晰且有帮助的。然而， 这种新方法从未在有限的研究环境之外进行过测试。 具体目标：测试模板匹配在比较 VA 不同急性病的护理质量方面的效用 护理医院，该项目将评估这种方法的可行性、准确性和可解释性。 具体来说，该项目将： (A1) 可行性：开发和优化两种模板匹配方法 比较 VA 各医院的 30 天死亡率。 (A2) Accuracy：比较模板匹配的能力 与传统回归法相比，可以正确识别表现不佳的医院。 (A3) 可解释性：比较 通过模板匹配生成的医院绩效数据的可解释性和可信度 与临床领导者的传统回归模型。 预期影响：该提案的总体目标是改善住院退伍军人的护理。 该提案将应用有前途的新模板匹配方法来对不同的 VA 进行基准测试 医疗保健系统并完成对这些方法的多方面评估。我们不仅会考虑 模板匹配的可行性和方法的严谨性，以及可解释性、可信度和 数据的问责制。我们预计模板匹配对于 VA 急性护理基准测试是可行的 医院，它将至少识别出表现不佳的医院以及当前的基准 传统的回归，对于 VA 医学主管来说，它比 当前的绩效报告。我们预计这项工作未来会有许多扩展，包括模板的使用 VA 与私营部门的比较匹配。 独特的特点和创新：该提案具有创新性，因为它将同时 (1) 推进 医院基准测试中最先进的统计数据，(2) 通过协商开发必要的基础设施 与运营合作伙伴一起在 VA 中使用模板匹配，并且 (3) 评估这种新方法，而不仅仅是 统计上，但也与它应该通知的实际用户有关。结果与 VA 高度相关，其 绩效始终受到审查。对于 VA 之外的许多医疗保健系统来说，它也引起了极大的兴趣 他们同样努力为各个医院的绩效提供有意义且可操作的评估。 项目方法：目标 1 将开发和完善 30 天死亡率基准的统计方法 在 VA 急症护理医院中使用两种模板匹配方法：(1) 单一模板和 (2) 个性化 每个医院的模板。将考虑多种统计方法进行匹配以实现最公平 各医院之间的比较。目标 2 将使用模拟和真实患者数据来衡量时间和原因 模板匹配和传统回归方法可能会产生不一致的医院评估。 目标 3 将开发模板匹配数据的呈现方式，然后调查 VA 医学主管进行比较 使用模板匹配与传统方法相比，绩效报告的可解释性和可信度 医院基准回归，并与部分酋长进行后续半结构化访谈 从医学酋长的角度进一步了解模板匹配的绩效报告。