Creating a Theoretical and Empirical Foundation for Better Non-Equivalent Control Group Designs in STEM Research

为 STEM 研究中更好的非等效对照组设计奠定理论和经验基础

• 批准号: 1760458
• 负责人: Thomas Cook
• 金额: $ 117.66万
• 依托单位: George Washington University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-08-15 至 2022-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1760458&HistoricalAwards=false
• 关键词: Creating; Theoretical; Empirical; Foundation; Better

This proposal was submitted in response to EHR Core Research (ECR) program announcement NSF 15-509. The ECR program of fundamental research in STEM education provides funding in critical research areas that are essential, broad and enduring. EHR seeks proposals that will help synthesize, build and/or expand research foundations in the following focal areas: STEM learning, STEM learning environments, STEM workforce development, and broadening participation in STEM. The ECR program is distinguished by its emphasis on the accumulation of robust evidence to inform efforts to (a) understand, (b) build theory to explain, and (c) suggest interventions (and innovations) to address persistent challenges in STEM interest, education, learning, and participation. Random assignment of a STEM innovation to participants (students, teachers, schools) affords the researcher the opportunity to estimate an unbiased estimate of the effect of an intervention. However, random assignment may not be possible for ethical and practical reasons. Consequently, it is often difficult to build statistical knowledge of the impact of a STEM innovation without introducing bias into the process. The research to be conducted here will build and test a framework that will allow researchers to minimize statistical bias and better understand the impacts of STEM innovations (e.g., a new curriculum, a new way of teaching, or a change in policy) in settings where random assignment is not possible. This proposal uses advances in statistical theory and past empirical findings to craft testable hypotheses that, if validated, will improve causal hypothesis testing in STEM education research. The main hypotheses speak to reducing, and perhaps eliminating, the selection bias in quasi-experiments. The proposal focuses on the bias-reducing role of three design elements: a comparison group that is local to the treatment group, a pretest measure of the study outcome, and a rich set of pre-intervention covariates that are multi-dimensional (cover more than one substantive domain), multi-temporal (cover more than one time point) and multi-level (available at both the student and school levels). Each of these elements often reduces some bias and sometimes eliminates it entirely. How do they combine to reduce bias? The proposal will test (1) which combinations of the elements reduces most bias; (2) how often the bias reduces to close to zero; (3) do they do so robustly across STEM datasets with diverse interventions and students; and (4) is bias inadvertently introduced, rather than reduced, when the 3 design elements are combined.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该提案是为了回应EHR核心研究（ECR）计划公告NSF 15-509提交。 STEM教育基础研究的ECR计划在重要，广泛和持久的关键研究领域提供资金。 EHR寻求将有助于综合，建立和/或扩大以下焦点领域的研究基础的建议：STEM学习，STEM学习环境，STEM劳动力发展以及扩大参与STEM。 ECR计划的特色是强调积累可靠的证据，以告知（a）理解，（b）建立解释理论，以及（c）提出干预措施（和创新），以应对STEM利益，教育的持续挑战，学习和参与。将STEM创新的随机分配给参与者（学生，教师，学校）使研究人员有机会估计对干预效果的公正估计。但是，出于道德和实际原因可能无法随机分配。因此，在不引入偏见的过程中，通常很难建立有关STEM创新影响的统计知识。这里要进行的研究将建立和测试一个框架，该框架将使研究人员能够最大程度地减少统计偏见，并更好地了解STEM创新的影响（例如，新课程，新的教学方式或政策变化）在设置中不可能随机分配。该建议使用统计理论和过去的经验发现的进步来制作可检验的假设，如果得到验证，将改善STEM教育研究中的因果假设检验。主要的假设表明了降低，甚至消除了准实验中的选择偏见。该提案的重点是三个设计元素的减少偏见作用：治疗组本地的比较组，对研究结果的预测量度量以及一组多维的干预前协变量（覆盖更多）比一个实体领域，多阶梯（覆盖一个以上的时间点）和多层次（在学生和学校级别上都可以使用）。这些元素中的每一个通常都会减少一些偏见，有时完全消除它。它们如何合并以减少偏见？该提案将测试（1）哪些元素组合减少了大多数偏见； （2）偏差多久减少到接近零的频率； （3）他们在各种干预措施和学生的STEM数据集中做到了强劲的做法； （4）当组合3个设计要素时，无意间引入而不是减少的偏见。该奖项反映了NSF的法定任务，并且使用基金会的知识分子优点和更广泛的审查标准，被认为值得通过评估来获得支持。