Collaborative Research: Designing Assessments in Physical Science Across Three Dimensions

合作研究：跨三个维度设计物理科学评估

• 批准号: 1903103
• 负责人: Christopher Harris
• 金额: $ 5.65万
• 依托单位: WestEd
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-09-01 至 2019-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1903103&HistoricalAwards=false
• 关键词: Collaborative; Research; Designing; Assessments; Physical

This is a collaborative proposal among the University of Illinois at Chicago, Michigan State University, and SRI International to develop, test, and analyze sets of technology-supported diagnostic classroom assessments for middle school (grades 6-8) physical science. Assessments are aligned with the performance assessment and evidence-centered design methodologies suggested in the Framework for K-12 Science Education (NRC, 2012). The study focuses on the development of new measures of learning that take into account the interdependence of science content and practice. Two disciplinary core ideas--Matter and its Interactions, and Energy--and two scientific and engineering practices--Constructing Explanations and Designing Solutions, and Developing and Using Models--are used for this purpose. The research questions are: (1) What are the characteristic features of science assessments based upon systematic application of the Evidence-Centered Design (ECD) assessment process?; (2) To what extent can assessment designs incorporate critical core idea, crosscutting concept and science/engineering practice dimensions in ways that both separate and integrate these dimensions as part of the design architecture?; (3) What is the evidence that the multiple dimensions of science learning (e.g., content, practices and crosscutting concepts) are separable and recoverable in the performance of students who respond to these assessments?; (4) How instructionally sensitive are these assessments? (i.e., Do they show differential and appropriate sensitivity to students' opportunity to learn science in ways consistent with the vision contained in the NRC Framework?); (5) What forms of evidence can be provided for the validity of these assessments using a multifaceted validity framework that takes into account both the interpretive and evidentiary components of a validity argument for these new assessments?; (6) What are the characteristics of assessments that best serve the needs of classroom teachers relative to a formative assessment process and in what ways do such assessments and scoring processes need to be designed to support effective teacher implementation?; and (7) What are the unique affordances and opportunities provided by technology in designing and implementing assessments focused on merging content & practices performance expectations? Assessments are iteratively designed and administered in three school districts and a laboratory school in Florida and one school district in Wisconsin using the "Investigating and Questioning our World through Science and Technology" curriculum. The three school districts in Florida have classrooms that are using typical curriculum. The assessments will also be administered and tested with students in these classrooms. To address the research questions, the project conducts five major tasks: (1) development of assessment items using the ECD process to document and guide coherence of items; (2) an alignment study to review design patterns and task templates; (3) a cognitive analysis study to empirically investigate the extent to which the items elicit the intended guidelines; (4) three empirical studies, including (a) an early-stage testing with teachers (n=6) and students (n=180) in Year 1, (b) a pilot testing in Year 2 with teachers (n=12) and students (n=360), and (c) a main study in Year 3 with teachers (n=30) and students (n=900); and (5) a study to investigate the formative use of the assessment items using teacher focus groups' feedback and analysis of student performance data from previous studies. Project outcomes are: (a) research-informed and field-tested assessment prototypes that measure students' thinking around the two physical science core ideas and the two scientific and engineering practices; (b) relevant data and procedures used in the studies; and (c) a framework for the formative use of the assessments, including guidelines, scoring rubrics, and criteria for assessment design decisions.

这是密歇根州立大学伊利诺伊大学和国际SRI International的合作提议，用于开发，测试和分析中学的技术支持的诊断课堂评估（6-8年级）物理科学。评估与K-12科学教育框架中建议的绩效评估和以证据为中心的设计方法一致（NRC，2012年）。该研究的重点是考虑科学内容和实践相互依存的新学习措施的发展。两个纪律核心思想 - 及其相互作用，以及两个科学和工程实践 - 构建解释和设计解决方案，开发和使用模型 - 用于此目的。研究问题是：（1）基于循证设计（ECD）评估过程的系统应用，科学评估的特征是什么？ （2）评估设计可以在多大程度上结合关键的核心思想，跨越概念和科学/工程实践维度，以分开并整合这些维度作为设计体系结构的一部分？ （3）什么是什么证据表明，科学学习的多个维度（例如，内容，实践和交叉概念）在对这些评估做出回应的学生的表现中是可分开且可回收的？ （4）这些评估对教学的敏感程度如何？ （即，他们是否表现出对学生学习科学的机会，以与NRC框架中包含的愿景一致的方式显示出差异化和适当的敏感性？）； （5）使用多方面的有效性框架可以为这些评估的有效性提供哪些形式的证据，该框架考虑了这些新评估的有效性论点的解释性和证据成分？ （6）相对于形成性评估过程，最能满足课堂教师需求的评估的特征是什么？需要设计这些评估和评分过程以支持有效的教师实施？ （7）技术在设计和实施评估方面提供了什么独特的负担和机会，重点是合并内容和实践绩效期望？评估是在佛罗里达州的三个学区和一所实验室的迭代设计和管理的，威斯康星州的一个学区使用“通过科学技术调查和质疑我们的世界”课程。佛罗里达州的三个学区都有使用典型课程的教室。评估还将与这些教室的学生进行管理和测试。为了解决研究问题，该项目执行了五项主要任务：（1）使用ECD流程来记录和指导项目连贯性的评估项目； （2）一项对齐研究，以审查设计模式和任务模板； （3）一项认知分析研究，以经验研究项目引发预期准则的程度； （4）三项经验研究，包括（a）1年级的教师（n = 6）和学生（n = 180）的早期测试，（b）在2年级进行的试点测试（n = 12）和学生（n = 360）和（c）（c）在3年级的主要研究（c），与老师（n = 30）和学生（n = 900）进行了一项主要研究； （5）一项研究，使用教师焦点小组的反馈和对以前研究的学生绩效数据的分析来研究评估项目的形成性使用。 项目成果是：（a）经过研究的和现场测试的评估原型，这些原型衡量了学生围绕两个物理科学核心思想以及两种科学和工程实践的思考； （b）研究中使用的相关数据和程序； （c）用于形成评估的框架，包括指南，评分标准和评估设计决策标准。