Empirical Benchmarks of Design Parameters for Group Randomized Trials in Teacher Professional Development Intervention Studies

教师专业发展干预研究中分组随机试验设计参数的实证基准

• 批准号: 1405601
• 负责人: Benjamin Kelcey
• 金额: $ 47.6万
• 依托单位: University of Cincinnati Main Campus
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2013
• 资助国家: 美国
• 起止时间: 2013-08-15 至 2016-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1405601&HistoricalAwards=false
• 关键词: Empirical; Benchmarks; Design; Parameters; Group

Professional development of teachers is a strong component in the theory of action of improving student outcomes, yet such efforts are rarely rigorously evaluated. One of the problems of robust evaluation of professional development is that these programs are largely conducted with teachers in their school or district settings rather than being randomly assigned individually to different activities. This requires that the evaluation design be developed around group randomized trials, and the field of educational evaluation lacks important knowledge of how to structure such evaluations. This project is developing estimates of the empirical benchmarks of design parameters to be used to structure evaluations of whole school professional development efforts in mathematics. The project uses the large data sets that have been accumulated using the Mathematical Knowledge of Teaching (MKT) measures to develop estimates of the variation in teacher pedagogical content knowledge in mathematics to provide values that can be used to estimate sample size and other design parameters in evaluation design. Researchers from the College of Education at Wayne State University, the Educational Testing Service and Michigan State University are conducting this research.This project leverages the data collected from multiple professional development projects for mathematics teachers that have used the MKT under the auspices of the researchers that have been collected in an online Teacher Knowledge Assessment System from over 10,000 teachers in over 1150 schools. The data from the MKT is combined with data that is available publically in the Common Core of Data about the schools and districts within which the professional development was conducted and additional data that have been collected by the MKT researchers. The researchers use random intercept hierarchical linear models to estimate the design parameters for four different group random trial designs. They examine the impact of several covariate sets of parameters on the variance components and the corresponding estimates of intraclass correlation. The resulting parameters are reported in association with the conditions of the schools and districts in which the data were generated.The findings from this study are reported in the professional literature on research and evaluation design. In addition, professional development workshops disseminate the findings and the processes used to develop them at national educational research and evaluation meetings. In addition, the researchers include their findings in updates of the Optimal Design software that is used by multiple researchers and evaluators to structure the design of their own studies. Too frequently, researchers do not have sufficient knowledge of the potential contextual issues surrounding the schools and districts in which they work to make accurate estimates of the parameters that they should use to determine important characteristics of their research such as sample size. The findings of this study provide more realistic estimates of the design parameters that researchers and evaluators need to conduct their own studies.

教师的专业发展是改善学生成果的行动理论的重要组成部分，但是很少对这些努力进行严格的评估。对专业发展评估的强大评估问题之一是，这些计划主要是与学校或地区环境中的教师一起进行的，而不是被单独分配给不同的活动。这就要求围绕小组随机试验进行评估设计，并且教育评估领域缺乏有关如何构建此类评估的重要知识。该项目正在开发对设计参数的经验基准的估计值，以构建整个学校数学专业发展工作的评估。该项目使用使用数学知识（MKT）措施积累的大数据集来开发数学中教师教学内容知识变化的估计，以提供可用于估算样本量和其他设计参数的值评估设计。 韦恩州立大学，教育测试服务和密歇根州立大学的教育学院的研究人员正在进行这项研究。该项目利用了从多个专业发展项目中收集的数据，用于数学教师，这些项目在研究人员的奥斯基克斯（MKT）的研究中使用了MKT已从1150多所学校的10,000多名教师中收集了在线教师知识评估系统中。来自MKT的数据与有关进行专业发展的学校和地区的共同核心公开可用的数据结合使用，并由MKT研究人员收集的其他数据。研究人员使用随机拦截层次线性模型来估计四个不同组随机试验设计的设计参数。他们检查了几个协变量参数对方差成分的影响以及类内相关的相应估计值。报告的参数与产生数据的学校和地区的条件有关。该研究的发现是在研究和评估设计的专业文献中报告的。此外，专业发展研讨会在国家教育研究和评估会议上传播了用于开发它们的发现和过程。此外，研究人员还将其发现包括在最佳设计软件的更新中，该软件由多个研究人员和评估人员使用来构建自己的研究的设计。太常见了，研究人员对围绕学校和地区的潜在上下文问题没有足够的了解，以准确估算他们应使用的参数来确定其研究的重要特征，例如样本量。这项研究的发现提供了对研究人员和评估人员进行自己的研究所需的设计参数的更现实的估计。