Developing an Unbiased Machine Learning Tool for Prediction of Acute Coronary Syndrome

开发用于预测急性冠状动脉综合征的无偏差机器学习工具

基本信息

批准号：
10258045
负责人：
Qingqing Mao
金额：
$ 25.66万
依托单位：
DASCENA, INC.
依托单位国家：
美国
项目类别：
财政年份：
2021
资助国家：
美国
起止时间：
2021-09-18 至 2022-03-31
项目状态：
已结题

来源：
https://reporter.nih.gov/project-details/10258045
关键词：
Acute Coronary Event Address Adoption Age Area Artificial Intelligence California Clinical Data Data Analyses Data Collection Data Set Detection Development Diagnosis Early Diagnosis Electronic Health Record Emergency Department patient Ensure Evaluation Fatigue Feedback Female Future Gender Hospitals Individual Inpatients Lead Length of Stay Machine Learning Measurement Measures Methods Mind-Body Method Modeling Myocardial Infarction Outcome Patient-Focused Outcomes Patients Performance Physicians Platelet Glycoprotein GPIIb-IIIa Complex Platelet Glycoproteins Play Population Probability ROC Curve Race Registries Research Resource Allocation Risk Role SCAP2 gene San Francisco Scheme Severities Sex Bias Sex Differences System Technology Testing Time Training Trees Universities Unstable angina Weight Woman Work acute care acute coronary syndrome age group algorithm development algorithm training base classifier algorithm clinical decision support clinical decision-making clinical practice design electronic data eptifibatide health care disparity health care settings health inequalities live stream machine learning algorithm male men mortality overtreatment prediction algorithm predictive modeling predictive tools prospective racial bias racial disparity receptor resource guides response risk stratification sex sex disparity standard of care support tools thrombolysis tool trait

项目摘要

Abstract Significance: Racial and sex disparities in the diagnosis and care of acute coronary syndrome (ACS) patients are well documented. As machine learning algorithms (MLA) become more common in healthcare settings, it is imperative to ensure that these methods do not contribute to disparities through biased predictions or differential accuracy across racial and sex groups. Research Question: Can a MLA be trained to be more accurate and less biased than commonly used risk stratification systems for ACS prediction? Prior Work: The research team developed a preliminary gradient boosted tree model for myocardial infarction (MI) prediction using retrospective data from electronic health records. On a hold-out test set, the algorithm classifier attained an area under the receiver operating characteristic curve (AUROC) value of 0.92 when tested for the detection of MI at any point during a patient’s hospital stay. Other prior work by the research team involved development of a MLA to minimize bias in inpatient mortality predictions between White and non-White patient groups. The model was found to be unbiased as measured by the equal opportunity difference (EOD = 0.016, p = 0.204) and outperformed commonly used severity scoring systems MEWS, SAPS-II, and APACHE in respect to bias and accuracy. Specific Aims: In Aim 1, an unbiased model for early ACS prediction will be developed. Preprocessing the MLA training data will remove aspects of the data that reflect systemic health inequities while maintaining the aspects of the data that reflect relevant patient measurements and outcomes. Assessment of equal opportunity difference (EOD) and the Zemel statistic will provide a means to evaluate the MLA’s ability to operate without sex or racial bias. In Aim 2, the model’s performance will be compared to three commonly used ACS risk stratification scores. Evaluating model performance and bias against these systems will allow for comparison of the unbiased MLA to the current ACS standard of care. Methods: Aim 1: An ACS prediction algorithm that will be demonstrated to be unbiased when comparing performance accuracy on White vs. non-White and male vs. female emergency department patients will be developed. The model’s performance will be assessed with regard to the EOD and Zemel statistic, which measure the difference in false negative results and average predicted risk, respectively, between White and non-White and male and female patients under the null hypothesis of no difference. Aim 2: Model performance will be compared to modified versions of three other commonly used ACS risk stratification scores: the Global Registry of Acute Coronary Events (GRACE) score; the Platelet glycoprotein IIb/IIIa in Unstable angina: Receptor Suppression Using Integrilin (eptifibatide) Therapy (PURSUIT) score; and the Thrombolysis in Myocardial Infarction (TIMI) score, some of which have been shown to perform differentially across gender and race. EOD and the Zemel statistic will also be assessed as a measure of bias for the MLA, GRACE, PURSUIT and TIMI scores. Future Directions: The MLA will be implemented in live hospital settings for prospective evaluation.

抽象的意义：急性冠脉综合征 (ACS) 患者诊断和护理中的种族和性别差异随着机器学习算法 (MLA) 在医疗保健环境中变得越来越普遍，它已得到充分记录。必须确保这些方法不会通过有偏见的预测或结果造成差异不同种族和性别群体的准确性存在差异研究问题：MLA 是否可以通过训练变得更加准确。比常用的 ACS 预测风险分层系统更准确且偏差更小？研究团队开发了用于心肌梗塞（MI）预测的初步梯度提升树模型使用来自电子健康记录的回顾性数据在保留测试集上，算法分类器获得了结果。测试时，受试者工作特征曲线下面积 (AUROC) 值为 0.92 研究小组之前所做的其他工作包括在患者住院期间的任何时间检测心肌梗死。制定 MLA，以最大限度地减少白人和非白人之间住院患者死亡率预测的偏差通过平等机会差异（EOD =）衡量，该模型被发现是无偏见的。 0.016，p = 0.204）并且优于常用的严重性评分系统 MEWS、SAPS-II 和 APACHE 关于偏差和准确性：在目标 1 中，早期 ACS 预测的无偏差模型将是开发的 MLA 训练数据的预处理将删除反映系统健康状况的数据的某些方面。不平等，同时保持反映相关患者测量和结果的数据方面。平等机会差异 (EOD) 和 Zemel 统计数据的评估将提供一种评估机会差异的方法。 MLA 在没有性别或种族偏见的情况下运作的能力在目标 2 中，模型的表现将与三个模型进行比较。常用的 ACS 风险分层评分评估模型性能和针对这些系统的偏差。将允许将无偏见的 MLA 与当前 ACS 护理标准进行比较方法：目标 1：ACS。在比较白棋表现准确性时将证明预测算法是无偏的将开发急诊科患者与非白人、男性与女性患者的模型。性能将根据 EOD 和 Zemel 统计数据进行评估，这些统计数据衡量白人与非白人、男性与非白人之间的假阴性结果和平均预测风险目标 2：将模型性能与无差异的零假设下的女性患者进行比较。其他三个常用 ACS 风险分层评分的修改版本：全球急性病登记处冠状动脉事件 (GRACE) 评分；不稳定型心绞痛中的血小板糖蛋白 IIb/IIIa：受体抑制使用 Integrilin（依替巴肽）治疗 (PURSUIT) 评分和心肌梗塞溶栓 (TIMI)；分数，其中一些已被证明在性别和种族方面表现有所差异。统计数据还将被评估为 MLA、GRACE、PURSUIT 和 TIMI Future 分数的偏差衡量标准。说明：MLA 将在现场医院环境中实施以进行前瞻性评估。