Improving Reproducibility by Incorporating Uncertainty

通过纳入不确定性来提高再现性

• 批准号: 10322751
• 负责人: Timothy L. Lash
• 金额: $ 27.05万
• 依托单位: EMORY UNIVERSITY
• 依托单位国家: 美国
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-02-06 至 2024-01-31
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/10322751
• 关键词: Address; Appearance; Area; Bayesian Method; Behavior; Body mass index; CYP2D6 gene; Clinical; Communities; Consensus; Creativeness; Data Collection; Effectiveness; Evaluation; Genotype; Grant; Guidelines; Heterogeneity; Individual; Influentials; Informatics; Journals; Judgment; Life Style; Measurement; Meta-Analysis; Methods; Obesity; Outcome; Paper; Perception; Predisposition; Publication Bias; Reaction; Relative Risks; Reporting; Reproducibility; Research; Research Personnel; Residual state; Resources; Risk; Selection Bias; Source; Speed; Systematic Bias; Tamoxifen; Techniques; Testing; Uncertainty; United States National Institutes of Health; advanced analytics; base; cancer recurrence; complex data; disorder later incidence prevention; editorial; evidence base; improved; informatics tool; malignant breast neoplasm; metabolic phenotype; mortality; simulation; skills; web-enabled

In recent years, stakeholders in the scientific and lay communities have raised alarms about a lack of reproducibility of scientific results. These stakeholders view the reproducibility crisis as a product of the behavior of researchers and editors. While these behaviors likely have an impact on reproducibility, there are credible reasons why studies of complex data should be expected to arrive at different estimates. First, each study is differentially susceptible to systematic biases, including confounding, selection bias and measurement error. These biases may be large drivers of the appearance of poor reproducibility. Second, many studies that have been criticized for lack of replication are small, and therefore subject to substantial random variability. In combination with selection forces emanating from significance testing, these small studies are likely to overestimate effects, further contributing to the appearance of poor reproducibility. To date, proposed solutions to the perceived reproducibility crisis have largely ignored these contributing factors. We propose to (a) use simulation-based quantitative bias analysis techniques to adjust for the influence of systematic errors on estimates of association and on summaries of an evidence base, and (b) use Bayesian statistical methods to synthesize prior information with estimates of association and summaries of an evidence base to reduce random variability. The premise of the proposed project is that the use of these informatics approaches will reduce the potential for systematic and random error to misleadingly portray research as poorly reproducible and will identify the most important limitations in an evidence base, which will optimize decisions regarding new data collection. The proposed informatics will be extended and applied in the context of two high profile, controversial topic areas with complex data, which will provide examples applicable to other topic areas. For both topic areas, many sources of potential bias have been identified in the surrounding discourse—as have powerful sources of prior information to temper uncertainty—but their influences on individual estimates of association and summaries of the evidence base have not been fully quantified. Quantitative adjustments for these errors using quantitative bias analysis and Bayesian methods—and for publication bias on meta-analytic summaries—would improve reproducibility. We will then extend and apply web-enabled informatics tools to implement the methods for any topic with a set of heterogeneous study results, allowing stakeholders without advanced analytic skills to tailor the underlying assumptions and see for themselves the impact on the summary results. By achieving our aims, this project will advance the use of research informatics to diminish the reproducibility crisis, help to speed consensus-building for any research topic, and productively channel research resources towards resolving the most influential sources of uncertainty in any topic area.

近年来，科学和外行社区的利益相关者发出了警报 科学结果的可重复性。这些利益相关者认为可重复性危机是 研究人员和编辑的行为。虽然这些行为可能会影响繁殖，但仍有 应预期对复杂数据的研究的可信原因将得出不同的估计。首先，每个 研究对系统偏见的影响不同，包括混杂，选择偏见和测量 错误。这些偏见可能是可重复性差的大型驱动因素。其次，许多研究 对于缺乏复制而言至关重要，因此要经过实质性的随机变异性。在 与显着性测试中发出的选择力相结合，这些小型研究很可能 高估了效果，进一步导致了可重复性差的出现。迄今为止，拟议的解决方案 对于感知到的可重复性危机而言，这些因素在很大程度上忽略了这些因素。我们建议（a）使用 基于仿真的定量偏差分析技术可以调整系统错误对 估计关联和证据基础的摘要，以及（b）使用贝叶斯统计方法 合成先验信息，并估算关联和证据基础的摘要以减少 随机变异性。拟议项目的前提是使用这些信息效果方法将 降低系统和随机错误的潜力，以误导性地将研究描绘成可再现不佳的研究 并将确定证据基础中最重要的限制，该局限性将优化有关新的决定 数据收集。拟议的信息将在两个备受瞩目的背景下扩展和应用 具有复杂数据的有争议的主题领域将提供适用于其他主题领域的示例。为了 这两个主题领域，在周围的话语中都发现了许多潜在偏见的来源 - 有力的先前信息来源到温度不确定性，但它们对个人估计的影响 证据基础的关联和摘要尚未完全量化。定量调整 这些错误使用定量偏置分析和贝叶斯方法的错误，以及用于元分析的出版偏见 摘要 - 将提高可重复性。然后，我们将扩展并应用启用网络的信息工具 通过一组异构研究结果实施任何主题的方法，使利益相关者没有 先进的分析技能来量身定制基本假设，并亲眼目睹对 摘要结果。通过实现我们的目标，该项目将推动使用研究信息来减少 可重复性危机，有助于加快任何研究主题的共识，并有效地渠道 研究资源以解决任何主题领域中最具影响力的不确定性来源。

项目成果

Yland et al. Respond to "Heuristics and Wish Bias".

伊兰等人。

• DOI: 10.1093/aje/kwac092
• 发表时间: 2022
• 期刊: American journal of epidemiology
• 影响因子: 5
• 作者: Yland,JenniferJ;Wesselink,AmeliaK;Lash,TimothyL;Fox,MatthewP
• 通讯作者: Fox,MatthewP

SAS and R code for probabilistic quantitative bias analysis for misclassified binary variables and binary unmeasured confounders.

用于错误分类二元变量和二元未测量混杂因素的概率定量偏差分析的 SAS 和 R 代码。

• DOI: 10.1093/ije/dyad053
• 发表时间: 2023
• 期刊: International journal of epidemiology
• 影响因子: 7.7
• 作者: Fox,MatthewP;MacLehose,RichardF;Lash,TimothyL
• 通讯作者: Lash,TimothyL

Application of the Adaptive Validation Substudy Design to Colorectal Cancer Recurrence.

适应性验证子研究设计在结直肠癌复发中的应用。

• DOI: 10.2147/clep.s230314
• 发表时间: 2020
• 期刊: Clinical epidemiology
• 影响因子: 3.9
• 作者: Collin,LindsayJ;Riis,AndersH;MacLehose,RichardF;Ahern,ThomasP;Erichsen,Rune;Thorlacius-Ussing,Ole;Lash,TimothyL
• 通讯作者: Lash,TimothyL

Opioid-related Mortality in United States Death Certificate Data: A Quantitative Bias Analysis With Expert Elicitation of Bias Parameters.

美国死亡证明数据中与阿片类药物相关的死亡率：专家提取偏差参数的定量偏差分析。

• DOI: 10.1097/ede.0000000000001600
• 发表时间: 2023
• 期刊: Epidemiology (Cambridge, Mass.)
• 作者: Goldsmith,ElizabethS;Krebs,ErinE;Ramirez,MarizenR;MacLehose,RichardF
• 通讯作者: MacLehose,RichardF

Practical data considerations for the modern epidemiology student.

现代流行病学学生的实用数据注意事项。

• DOI: 10.1016/j.gloepi.2021.100066
• 发表时间: 2021
• 期刊: Global epidemiology
• 作者: Tran,NguyenK;Lash,TimothyL;Goldstein,NealD
• 通讯作者: Goldstein,NealD