The Internal Validity of External Validity: Using Experiments to Validate Three Approaches to Extrapolating Causal Inferences Beyond the Cutoff in Regression Discontinuity

外部有效性的内部有效性：使用实验验证三种方法来推断超出回归不连续性截止值的因果推论

• 批准号: 1544301
• 负责人: Thomas Cook
• 金额: $ 79.46万
• 依托单位: MATHEMATICA POLICY RESEARCH INC
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2015
• 资助国家: 美国
• 起止时间: 2015-10-01 至 2018-12-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1544301&HistoricalAwards=false
• 关键词: Internal; Validity; External; Using; Experiments

The Promoting Research and Innovation in Methodologies for Evaluation (PRIME) program seeks to support research on evaluation with special emphasis on: (1) exploring innovative approaches for determining the impacts and usefulness of STEM education projects and programs; (2) building on and expanding the theoretical foundations for evaluating STEM education and workforce development initiatives, including translating and adapting approaches from other fields; and (3) growing the capacity and infrastructure of the evaluation field. Three types of proposals will be supported by the program: Exploratory Projects that include proof-of-concept and feasibility studies; more extensive Full-Scale Projects; and workshops and conferences. The proposed research attends carefully to item 1 above. In addition, the dissemination plan also attends to item 3. Randomized control trials in STEM education are considered the gold standard for causal inference. Often these models are overly restrictive in practice (for ethical and other reasons). Moreover, many such studies collapse as the research progresses. For example, senior school leadership changes and a school is withdrawn from a study. When this occurs the randomizing process is compromised and so are the resulting inferences. The proposed research will assess the validity and robustness of new approaches for using the regression discontinuity design (RDD) to evaluate educational interventions affecting STEM outcomes. This study will systematically compare and contrast the new approaches for generalizing RDD, assessing how well they produce unbiased causal estimates away from the cutoff score. The research will be conducted using 12 heterogeneous datasets from past educational interventions that (a) were originally evaluated through an RE and (b) included math achievement as an outcome measure. Using the within-study comparison method (Cook et al. 2008), this study will assess the extent to which the RE and a given RDD approach produce similar causal estimates away from the cutoff for the same population. Analyzing multiple datasets with STEM outcomes will provide information about the robustness of the new RDD methods across different types of interventions and student populations.

促进评估方法（PRIME）计划的研究和创新旨在支持评估研究，并特别强调：（1）探索确定STEM教育项目和计划的影响和实用性的创新方法； （2）建立和扩大理论基础，以评估STEM教育和劳动力发展计划，包括来自其他领域的翻译和适应方法； （3）增强评估领域的能力和基础设施。该计划将支持三种建议：探索性项目，包括概念证明和可行性研究；更广泛的全面项目；以及研讨会和会议。拟议的研究仔细地参加了上面的项目1。此外，传播计划还参与了第3项。STEM教育中的随机对照试验被认为是因果推断的黄金标准。 通常，这些模型在实践中过于限制（出于道德和其他原因）。此外，许多这样的研究随着研究的进行而崩溃。例如，从一项研究中撤回了高中领导力的变化和学校。 发生这种情况时，随机过程被损害，由此产生的推论也是如此。拟议的研究将评估使用回归不连续设计（RDD）的新方法的有效性和鲁棒性，以评估影响STEM结果的教育干预措施。这项研究将系统地比较和对比概括RDD的新方法，评估它们与截止评分相比产生无偏的因果估计值。这项研究将使用过去的12个非均质数据集进行（a）最初通过RE评估（a），并且（b）将数学成就作为结果指标进行评估。使用研究内比较方法（Cook等，2008），本研究将评估RE和给定的RDD方法在同一人群中产生类似的因果估计的程度。分析具有STEM结果的多个数据集将提供有关在不同类型的干预措施和学生群体中新的RDD方法鲁棒性的信息。