BCSER: Developing Expertise in Educational Testing to Support Characterizing the Impact of Organic Chemistry Instruction on Students' Ability to Reason with Representations

BCSER：发展教育测试方面的专业知识，以支持表征有机化学教学对学生用表征进行推理的能力的影响

• 批准号: 2025216
• 负责人: Maia Popova
• 金额: $ 34.46万
• 依托单位: University of North Carolina Greensboro
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-10-01 至 2023-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2025216&HistoricalAwards=false
• 关键词: BCSER; Developing; Expertise; Educational; Testing

Representational competence is the ability to use drawings, diagrams, and symbols to learn, communicate, and solve problems in a specific area, such as organic chemistry. Studies have shown that undergraduate students’ ability to reason with specific representations is important for their understanding of and success in organic chemistry. However, no validated assessment instruments currently exist that measure students’ representational competence in organic chemistry. With support from NSF’s EHR Core Research Building Capacity in STEM Education Research (ECR: BCSER) program, this project seeks to develop such an instrument. In the process of conducting this project, the principal investigator will develop competencies in educational testing and psychometrics. These new competencies will complement existing expertise in qualitative investigation of students’ understanding of chemistry concepts and characterization of instructors’ teaching beliefs and classroom practices. The high-quality assessment instrument developed by the project team is expected to be useful for further research on organic chemistry learning, as well as for helping college chemistry instructors make informed decisions about their teaching.This study will use a combination of classical test theory and factor analysis methods to develop and test the Organic-chemistry Representational Competence Assessment (ORCA). ORCA will measure student representational competence skills in the context of organic chemistry representations such as dash-wedge diagrams, Newman projections, and chair conformations. The targeted skills to be measured by ORCA include: the ability to (a) interpret these representations, (b) select an appropriate representation for a particular purpose, (c) make connections across these representations, and (d) use these representations to make predictions about chemical phenomena. The measurement quality framework, which targets select psychometric evidence informed by the Standards for Educational and Psychological Testing, will be used as the conceptual and methodological framework for evaluating the quality of ORCA’s items and its capacity to assist in drawing valid and reliable inferences about student representational competence skills. The instrument will be rigorously evaluated and tested with organic chemistry students from the University of North Carolina at Greensboro and the University of South Florida. The principal investigator will be mentored by an expert in the area of assessment of content in undergraduate chemistry courses, and will complete coursework to develop proficiency in psychometrics and measurement. The success of the project and achievement of project objectives will be assessed through periodic meetings with both the mentor and the advisory board. Dissemination of project work will be facilitated via publications in academic journals, presentations at relevant conferences, and workshops on assessment measurement design and evaluation. Results of this study will inform instructional practices and evidence-based design of organic chemistry curricula that supports the use of effective strategies to develop student representational competence. This project is supported through the EHR Core Research: Building Capacity in STEM Education Research (ECR: BCSER) competition that is designed to build individuals’ capacity to carry out high-quality, fundamental STEM education research in STEM learning and learning environments, broadening participation in STEM fields, and STEM workforce development.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

表征能力是指使用绘图、图表和符号来学习、交流和解决特定领域（例如有机化学）问题的能力。研究表明，本科生使用特定表征进行推理的能力对于他们理解事物很重要。然而，目前还没有经过验证的评估工具来衡量学生在有机化学方面的代表性能力，在 NSF 的 EHR STEM 教育研究核心研究能力 (ECR: BCSER) 计划的支持下，该项目旨在开发这样的评估工具。在开展该项目的过程中，首席研究员将培养教育测试和心理测量方面的能力，这些新能力将补充学生对化学概念的理解以及教师教学信念和课堂实践的定性调查方面的现有专业知识。项目团队开发的高质量评估工具预计将有助于有机化学学习研究，以及帮助大学化学教师做出明智的教学决策。本研究将进一步结合经典测试理论和因子分析方法来开发和测试有机化学表征能力评估 (ORCA) 将衡量学生在有机化学表征方面的表征能力技能，例如破折号楔形图、纽曼投影和椅子构象。 ORCA 测量的目标技能包括： (a) 解释这些表示，(b) 为特定目的选择适当的表示，(c) 在这些表示之间建立联系，以及 (d) 使用这些表示来预测化学现象。其目标是根据教育和心理测试标准选择心理测量证据，将用作评估 ORCA 项目质量的概念和方法框架，及其协助对学生表达能力技能做出有效和可靠推论的能力。由北卡罗来纳大学格林斯博罗分校和南佛罗里达大学的有机化学学生进行严格的评估和测试，首席研究员将在本科化学课程内容评估领域的专家的指导下完成课程作业。到培养心理测量学和测量方面的熟练程度。项目的成功和项目目标的实现将通过与导师和顾问委员会的定期会议进行评估。项目工作的传播将通过学术期刊上的出版物、相关会议上的演讲来促进。这项研究的结果将为有机化学课程的教学实践和循证设计提供信息，支持使用有效的策略来培养学生的表达能力。该项目得到了 EHR 核心研究的支持：构建。 STEM 教育研究能力（ECR：BCSER）竞赛旨在培养个人在 STEM 学习和学习环境中开展高质量、基础 STEM 教育研究的能力，扩大对 STEM 领域的参与以及 STEM 劳动力发展。该奖项通过使用基金会的智力价值和更广泛的影响审查标准进行评估，NSF 的法定使命被认为值得支持。