Improving Reproducibility by Incorporating Uncertainty

通过纳入不确定性来提高再现性

基本信息

批准号：
10322751
负责人：
Timothy L. Lash
金额：
$ 27.05万
依托单位：
EMORY UNIVERSITY
依托单位国家：
美国
项目类别：
财政年份：
2019
资助国家：
美国
起止时间：
2019-02-06 至 2024-01-31
项目状态：
已结题

项目摘要

In recent years, stakeholders in the scientific and lay communities have raised alarms about a lack of reproducibility of scientific results. These stakeholders view the reproducibility crisis as a product of the behavior of researchers and editors. While these behaviors likely have an impact on reproducibility, there are credible reasons why studies of complex data should be expected to arrive at different estimates. First, each study is differentially susceptible to systematic biases, including confounding, selection bias and measurement error. These biases may be large drivers of the appearance of poor reproducibility. Second, many studies that have been criticized for lack of replication are small, and therefore subject to substantial random variability. In combination with selection forces emanating from significance testing, these small studies are likely to overestimate effects, further contributing to the appearance of poor reproducibility. To date, proposed solutions to the perceived reproducibility crisis have largely ignored these contributing factors. We propose to (a) use simulation-based quantitative bias analysis techniques to adjust for the influence of systematic errors on estimates of association and on summaries of an evidence base, and (b) use Bayesian statistical methods to synthesize prior information with estimates of association and summaries of an evidence base to reduce random variability. The premise of the proposed project is that the use of these informatics approaches will reduce the potential for systematic and random error to misleadingly portray research as poorly reproducible and will identify the most important limitations in an evidence base, which will optimize decisions regarding new data collection. The proposed informatics will be extended and applied in the context of two high profile, controversial topic areas with complex data, which will provide examples applicable to other topic areas. For both topic areas, many sources of potential bias have been identified in the surrounding discourse—as have powerful sources of prior information to temper uncertainty—but their influences on individual estimates of association and summaries of the evidence base have not been fully quantified. Quantitative adjustments for these errors using quantitative bias analysis and Bayesian methods—and for publication bias on meta-analytic summaries—would improve reproducibility. We will then extend and apply web-enabled informatics tools to implement the methods for any topic with a set of heterogeneous study results, allowing stakeholders without advanced analytic skills to tailor the underlying assumptions and see for themselves the impact on the summary results. By achieving our aims, this project will advance the use of research informatics to diminish the reproducibility crisis, help to speed consensus-building for any research topic, and productively channel research resources towards resolving the most influential sources of uncertainty in any topic area.

近年来，科学界和非专业界的利益相关者对缺乏科学知识提出了警告。这些利益相关者将可重复性危机视为科学结果的可重复性的产物。研究人员和编辑的行为虽然可能会对再现性产生影响，但也存在一些问题。为什么复杂数据的研究应该得出不同的估计值的令人信服的理由首先，每个。研究不同程度地容易受到系统偏差的影响，包括混杂、选择偏差和测量这些偏差可能是导致再现性差的主要原因。因缺乏重复性而受到批评的样本量很小，因此受到很大的随机变异性的影响。结合显着性检验产生的选择力，这些小型研究可能会高估了效果，进一步导致了再现性差的出现。迄今为止，提出的解决方案。对于所感知的可重复性危机，我们建议（a）使用这些影响因素。基于模拟的定量偏差分析技术，用于调整系统误差对关联估计和证据基础摘要，以及 (b) 使用贝叶斯统计方法将先验信息与关联估计和证据基础摘要相结合，以减少拟议项目的前提是这些信息学方法的使用将减少系统性和随机性错误的可能性，从而误导性地将研究描述为重复性差并将确定证据库中最重要的限制，这将优化有关新产品的决策拟议的信息学将在两个备受瞩目的背景下得到扩展和应用。具有复杂数据的有争议的主题领域，这将提供适用于其他主题领域的示例。在这两个主题领域，在周围的话语中已经发现了许多潜在偏见的来源——正如先验信息的强大来源可以缓和不确定性，但它们对个人估计的影响基础证据的关联和总结尚未完全量化。使用定量偏差分析和贝叶斯方法以及荟萃分析的发表偏差来消除这些错误摘要 - 将提高可重复性，然后我们将扩展并应用支持网络的信息学工具。使用一组异构研究结果实施任何主题的方法，使利益相关者无需先进的分析技能来调整基本假设并亲自了解对结果的影响通过实现我们的目标，该项目将促进研究信息学的使用，以减少再现性危机，有助于加快任何研究主题的共识建立，并有效地引导用于解决任何主题领域中最具影响力的不确定性来源的研究资源。

项目成果

期刊论文数量（10）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

Common misconceptions about validation studies.

关于验证研究的常见误解。

DOI：
发表时间：
2020
期刊：
影响因子：
7.7
作者：
Fox, Matthew P;Lash, Timothy L;Bodnar, Lisa M
通讯作者：
Bodnar, Lisa M

Yland et al. Respond to "Heuristics and Wish Bias".

伊兰等人。

DOI：
发表时间：
2022-07-23
期刊：
影响因子：
5
作者：
Yland, Jennifer J;Wesselink, Amelia K;Lash, Timothy L;Fox, Matthew P
通讯作者：
Fox, Matthew P

Validation of LexisNexis Accurint in the Georgia Cancer Registry's Cancer Recurrence and Information Surveillance Program.

LexisNexis Accurint 在佐治亚州癌症登记处的癌症复发和信息监测计划中的验证。

DOI：
发表时间：
2021-05-01
期刊：
影响因子：
0
作者：
Woolpert, Kirsten M;Ward, Kevin C;England, Cameron V;Lash, Timothy L
通讯作者：
Lash, Timothy L

Misconceptions About the Direction of Bias From Nondifferential Misclassification.

关于非差异错误分类的偏差方向的误解。

DOI：
发表时间：
2022-07-23
期刊：
影响因子：
5
作者：
Yland, Jennifer J;Wesselink, Amelia K;Lash, Timothy L;Fox, Matthew P
通讯作者：
Fox, Matthew P

Application of the Adaptive Validation Substudy Design to Colorectal Cancer Recurrence.

适应性验证子研究设计在结直肠癌复发中的应用。

DOI：
发表时间：
2020
期刊：
影响因子：
3.9
作者：
Collin, Lindsay J;Riis, Anders H;MacLehose, Richard F;Ahern, Thomas P;Erichsen, Rune;Thorlacius;Lash, Timothy L
通讯作者：
Lash, Timothy L