ROC Curve Methodology

ROC曲线方法论

• 批准号: 8351177
• 负责人: Enrique F. Schisterman
• 金额: $ 18万
• 依托单位: EUNICE KENNEDY SHRIVER NATIONAL INSTITUTE OF CHILD HEALTH & HUMAN DEVELOPMENT
• 依托单位国家: 美国
• 资助国家: 美国
• 起止时间: 至
• 项目状态: 未结题
• 来源: https://reporter.nih.gov/project-details/8351177
• 关键词: Accounting; Acute Disease; American; Area; Attenuated; Biological Assay; Biological Markers; Biomedical Research; Biotechnology; Chronic Disease; Computer software; Confidence Intervals; Data; Development; Devices; Diagnosis; Diagnostic; Diagnostic tests; Disease; Disease model; Early Diagnosis; Effectiveness; Evaluation; Individual; Infertility; Joints; Journals; Laboratories; Lead; Literature; Measurement; Measures; Methodology; Methods; Modeling; Normal Statistical Distribution; Normalcy; Outcome; Oxidative Stress; Patients; Peer Review; Performance; Population; Prevention; Probability; Process; Property; Publications; ROC Curve; Reaction; Receiver Operating Characteristics; Research; Research Personnel; Sampling; Screening procedure; Selection Bias; Source; Specificity; Specimen; Statistical Methods; Techniques; Testing; Work; base; cost; design; diagnostic accuracy; disease diagnosis; evidence base; improved; indexing; interest; novel diagnostics; population based; rapid growth; simulation; statistics; thiobarbituric acid; tool

Just as there are many markers of oxidative stress, the rapid growth of biotechnology means that researchers increasingly must consider which screening or diagnostic test to use in their research. My work with ROC curves is aimed at providing evidence-based approaches for making these choices. The ROC curve simultaneously plots the proportion of both abnormal and normal subjects correctly diagnosed at various test cutoff points. This graphical display facilitates the selection of an optimal threshold and enables easy comparison of the abilities of different tests. Increasingly, ROC curves are used in population based settings as opposed to settings where individuals have been pre-screened to some degree. However, ROC curve methods were not developed to account for common problems such as missing data, measurement error, linear combinations, confounding, referral bias, LODs, and other challenges. We have proposed estimators of the mean of a K-sample U-statistic (of which the area under the ROC curve (AUC) is a special case) when data on the outcomes of interest are missing in some sampled units and auxiliary variables are available in the entire sample. The proposed estimators exploit the information available in the auxiliaries without requiring assumptions about the joint distribution of the auxiliaries and outcomes. The properties of the proposed estimators are derived from general results on efficient semi-parametric estimation of the mean of a K-sample U-statistic with missing at random outcomes, observed auxiliary variables and known missingness probabilities. Random measurement error can attenuate a biomarkers ability to discriminate between diseased and non-diseased populations. We present an approach for estimating the Youden index, the AUC and its associated optimal cut-point for a normally distributed biomarker that corrects for normally distributed random measurement error. We also developed confidence intervals for these corrected estimates using the delta method and coverage probability through simulation of a variety of situations. Applying these techniques to the biomarker thiobarbituric acid reaction substance (TBARS), a measure of oxidative stress that has been proposed as a discriminating measurement for infertility, yields a 50% increase in diagnostic effectiveness at the optimal cut-point. This result may lead to biomarkers that were once naively considered ineffective becoming useful diagnostic devices. Since multiple markers are often available, we considered combining them to improve diagnostic accuracy. The linear combinations derived by Su and Liu (1993) that maximize the AUC may have unsatisfactorily low sensitivity over a certain range of desired specificity. We considered maximization of sensitivity over a range of specificity, and presented alternative linear combinations that have higher sensitivity over a range of high (or low) specificity. Additionally, we evaluated covariate effects on this linear combination assuming that the multiple markers or a transformation thereof, follow a multivariate normal distribution. We estimated the ROC curve of this linear combination of markers adjusted for covariates and approximate confidence intervals for the corresponding AUC. Another frequently encountered problem in studies that evaluate new diagnostic tests is that not all patients undergo disease verification due to the expense and/or invasiveness of the test. In fact, the decision to subject patients to verification testing often depends on the results of the new test and other predictors of disease status. For diagnostic tests where AUC estimation is based only on patients with verified disease status, the usual estimators are biased. We developed estimators that adjust for this bias. When information on disease status is missing, it is necessary either to model the missing data or the process leading to the missingness to obtain well-behavedestimators of the AUC. We have described a doubly robust estimator that is unbiased when the model for disease or the missingness is correct. This estimator does not require EM-type iterations and is easy to compute using standard software. It can accommodate both discrete and continuous markers and allows for the possibility that selection to verification is non-ignorable. In addition, the doubly robust estimator offers more protection against model misspecification than other currently available methods. We have applied the methods described above to show that TBARS, has discriminating abilities above and beyond chance. This work has yielded 23 publications in peer reviewed journals including Biometrika and the Journal of the American Statistical Association.

正如有许多氧化应激的标记一样，生物技术的快速生长意味着研究人员越来越必须考虑在其研究中使用哪种筛查或诊断测试。我使用ROC曲线的工作旨在提供基于证据的方法来做出这些选择。 ROC曲线同时绘制了在各种测试截止点正确诊断的异常和正常受试者的比例。此图形显示促进了最佳阈值的选择，并可以轻松比较不同测试的能力。 ROC曲线越来越多地用于基于人群的环境，而不是在某种程度上预先筛选个人的设置。但是，没有开发ROC曲线方法来解决常见问题，例如丢失的数据，测量误差，线性组合，混杂，转介偏见，LOD和其他挑战。 我们提出了关于K样本U统计量的平均值的估计量（其中ROC曲线下的区域（AUC）是特殊情况），当一些采样单位中缺少有关感兴趣结果的数据，并且整个样本中都有辅助变量。拟议的估计器利用辅助机构中可用的信息，而无需对辅助和结果的联合分布进行假设。所提出的估计量的性能是从一般的结果中得出的，对K-sample U统计数据的平均值有效的半参数估计，并且在随机结果下缺失，观察到的辅助变量和已知的缺失概率。 随机测量误差可能会削弱生物标志物区分患病和非疾病人群的能力。我们提出了一种估计Youden索引，AUC及其相关的最佳切点的方法，该方法是正态分布的生物标志物，该标志物可以纠正正态分布的随机测量误差。我们还使用DELTA方法和覆盖率概率通过模拟各种情况开发了这些校正后估计的置信区间。将这些技术应用于生物标志物硫巴比妥酸反应物质（TBARS），这是一种氧化应激的度量，已提出是对不育的区分测量，可在最佳切点时诊断有效性增加了50％。该结果可能导致生物标志物曾经被天真地认为无效成为有用的诊断设备。 由于通常可以使用多个标记，因此我们考虑将它们组合起来以提高诊断准确性。 Su and Liu（1993）得出的线性组合最大化AUC可能在一定的期望特异性范围内具有不令人满意的低灵敏度。我们考虑了在一系列特异性上的灵敏度的最大化，并提出了替代线性组合，在高（或低）特异性范围内具有较高灵敏度。此外，假设多个标记物或转换效果，我们评估了对这种线性组合的协变量效应，请遵循多元正态分布。我们估计了针对协变量调整的标记物的线性组合的ROC曲线，并为相应的AUC估计置信区间。 在评估新诊断测试的研究中，另一个经常遇到的问题是，并非所有患者都因测试的费用和/或侵入性而接受疾病验证。实际上，对患者进行验证测试的决定通常取决于新测试的结果和其他疾病状况的预测指标。对于AUC估计仅基于具有验证疾病状况的患者的诊断测试，通常的估计量是有偏见的。我们开发了调整这种偏见的估计器。 当缺少有关疾病状况的信息时，有必要对丢失的数据进行建模或导致失踪性的过程，以获得AUC的行为较高。我们已经描述了一个双重稳健的估计量，当疾病模型或缺失是正确的时，该估计量是公正的。该估计器不需要Em-type迭代，并且使用标准软件易于计算。 它可以容纳离散标记和连续标记，并允许选择验证是不可忽视的可能性。此外，比其他当前可用的方法相比，双重稳定的估计器对模型错误指定提供了更多的保护。 我们应用了上述方法，以表明TBARS具有超出机会的能力。这项工作已在同行评审的期刊上产生了23个出版物，其中包括生物埃特里卡和美国统计协会杂志。