"Improved Methodologies for Field Experiments: Maximizing Statistical Power While Promoting Replication."

“改进现场实验方法：在促进复制的同时最大化统计能力。”

• 批准号: 1461491
• 负责人: Michael Anderson
• 金额: $ 46万
• 依托单位: University of California-Berkeley
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2015
• 资助国家: 美国
• 起止时间: 2015-09-01 至 2019-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1461491&HistoricalAwards=false
• 关键词: Improved; Methodologies; Field; Experiments; Maximizing

This project proposes new methods for analyzing data from randomized controlled trials (RCTs), which are now increasingly used in some areas of economics and other social sciences. Researchers often want to use an RCT to test multiple different hypotheses. However, carrying out this kind of data analysis requires careful consideration of statistical issues; in particular, there is a chance that using the same data to run multiple different statistical analyses will increase the chance of false positives. As a result, economists conducting RCTs are increasingly specifying pre-analysis plans; that is, they are specifying how they plan to analyze the data before beginning the analysis. These plans, however, do limit researchers' ability to reuse data and learn from the data before beginning the statistical analysis. The research team will develop a potential mechanism for researchers to learn from the data without specifying every hypothesis in advance. They will use concepts from machine-learning as well as probability theory to provide a better framework for how to maximize power within a given RCT experiment. The results could therefore improve data analysis in important ways.The research team has evidence that in practice preanalysis plans include tests for many, even hundreds, of hypotheses. As a result current preanalysis plan designs are likely to be underpowered for many effects of economic interest. The team will extend and apply techniques from biostatistics, including gatekeeping, sequential testing, and FDR control to the design of preanalysis plans. They will also develop a split-sample approach in which researchers conduct exploratory analysis on one part of the data set, and, using estimates from that part of the data combined with theory over likely mechanisms, refine their hypotheses before registering a subset of hypotheses to be tested on the remaining part of the data.

该项目提出了分析来自随机对照试验（RCT）数据的新方法，这些方法现在越来越多地用于经济学和其他社会科学领域。 研究人员通常希望使用RCT检验多种不同的假设。 但是，进行这种数据分析需要仔细考虑统计问题。特别是，使用相同的数据运行多个不同的统计分析将增加误报的机会。 结果，进行RCT进行的经济学家越来越多地指定了整体分析计划。也就是说，他们正在指定他们计划在开始分析之前计划分析数据。 但是，这些计划确实会限制研究人员在开始统计分析之前重复使用数据并从数据中学习的能力。 研究团队将开发一种潜在的机制，以便研究人员在不提前指定所有假设的情况下从数据中学习。 他们将使用机器学习以及概率理论的概念来为如何在给定的RCT实验中最大化功率提供更好的框架。 因此，结果可以以重要的方式改善数据分析。研究团队有证据表明，在实践中，预先分析计划包括许多（甚至数百个假设）的测试。 结果，对于许多经济利益的影响，目前的PRENALISS PLAN设计设计可能会不足。 该团队将扩展并应用来自生物统计学的技术，包括守门，顺序测试和FDR控制，以设计用于预先分析计划的设计。他们还将开发一种分类方法，研究人员对数据集的一部分进行探索性分析，并使用从该数据的该部分结合理论而不是可能机制的估计值，然后在记录数据的其余部分进行测试的一部分假设之前，先改善其假设。