Application of Evidence-Centered Design to State Large-Scale Science Assessment

以证据为中心的设计在国家大规模科学评估中的应用

• 批准号: 0733172
• 负责人: Geneva Haertel
• 金额: $ 150万
• 依托单位: SRI International
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2007
• 资助国家: 美国
• 起止时间: 2007-09-01 至 2013-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0733172&HistoricalAwards=false
• 关键词: Application; Evidence; Centered; Design; State

The underlying claim of this proposal is that traditional test development processes have failed to produce science assessments that address high-reasoning levels, are not integrated with learning and formative assessments, and are too costly to implement at scale. It faces the challenge of conceptualizing and developing effective and affordable development processes that result in complex, interactive (e.g., simulations, investigations), valid, and standards-based science tasks that can be incorporated into existing statewide testing systems. In order to address this challenge, the proposal aims at (1) determining ways in which ECD enhances the quality of large-scale, technology-based assessments for middle school grades in key science education strands; (2) implementing resulting procedures in operational state-level test development; (3) evaluating the effectiveness and generalizability of these procedures, and (4) disseminating findings to the assessment community. The proposal builds on previously funded NSF research projects, such as Principled Assessment Designs for Inquiry?s (PADI), a REC-supported online system, which has developed data structures, representational forms, and software tools for the assessment of science inquiry concepts and skills (e.g., inquiry cycles, model-based reasoning) that allows efficient item development. The proposal also integrates the Evidence-Centered Design (ECD) methodology on which PADI is based. The ECD approach relates the learning to be assessed to an evidence model that specifies the student responses and scores serving as ?evidence of proficiency?. Further, the proposal integrates the Pearson Educational Measurement (PEM) test development infrastructure to create inquiry-based science tasks, and align those with the Minnesota Comprehensive Assessments. Specifically, the project aims at Anticipated products include (a) training materials for intended users, (b) ECD-based processes, (c) data structures and authoring interfaces, (d) exemplar design patterns and science tasks, (e) data analyses of large-scale administration of tasks (previously field- and pilot-tested), and (f) publications in scientific journals. Key questions on quality/validity, efficiency/costs, and technological benefits guide both formative and summative evaluation. Dissemination comprises a technical report series, presentations and workshops at professional organizations? annual meetings, and articles in peer-reviewed journals.

该提案的基本主张是，传统的测试开发过程未能进行科学评估，以解决高尚水平，与学习和形成性评估不整合，并且成本太高，无法大规模实施。它面临着概念化和开发有效且负担得起的开发过程的挑战，这些过程导致复杂，互动（例如，模拟，调查），有效和基于标准的科学任务，这些任务可以纳入现有的全州测试系统中。为了应对这一挑战，该提案的目的是（1）确定ECD提高基于技术的大型科学教育学院成绩的大规模，基于技术的评估的质量； （2）在运营状态级测试开发中实施结果程序； （3）评估这些程序的有效性和概括性，以及（4）向评估界传播发现。该提案建立在先前资助的NSF研究项目的基础上，例如针对询问的原则评估设计（PADI），这是一种由rec依支持的在线系统，该系统开发了数据结构，代表性形式和软件工具，用于评估科学探究概念和技能（例如，基于询问循环，基于模型的推理），可以进行有效的项目开发。该提案还整合了以证据为中心的设计（ECD）方法论。 ECD方法将学习的学习与一个证据模型相关联，该模型指定学生的反应和分数是“熟练程度的证据？此外，该提案还将皮尔逊教育测量（PEM）测试开发基础设施整合起来，以创建基于询问的科学任务，并与明尼苏达州的全面评估保持一致。具体而言，该项目旨在预期产品包括（a）针对预期用户的培训材料，（b）基于ECD的流程，（c）数据结构和创作界面，（d）示例设计模式和科学任务，（e）数据分析的数据分析（e）任务的大规模管理（以前的现场和驾驶员检验和（f）在科学上，以及（f）科学杂志。有关质量/有效性，效率/成本和技术福利的关键问题指导形成性和总结性评估。传播包括专业组织的技术报告系列，演讲和讲习班？年度会议和同行评审期刊的文章。