Collaborative Research: Improving the Effectiveness of Practice Exams in Large Introductory Physics Courses

协作研究：提高大型物理入门课程练习考试的有效性

• 批准号: 2021190
• 负责人: Jason Morphew
• 金额: $ 3.83万
• 依托单位: Purdue University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-09-01 至 2023-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2021190&HistoricalAwards=false
• 关键词: Collaborative; Research; Improving; Effectiveness; Practice

This project aims to serve the national interest by improving students' success in introductory STEM courses. For many college and university students, "gateway" courses, such as introductory physics, present a roadblock to further study in STEM fields. Every semester approximately 4,000 students take introductory courses in the Department of Physics at the University of Illinois. Most of the students are engineering majors, because physics is a required course for all programs in the College of Engineering. Despite instructors' use of evidence-based instructional methods such as online pre-lectures, active-learning lectures, "just-in-time teaching," peer instruction, and collaborative problem-solving in recitation sections, nearly 15 percent of students do not pass the introductory courses. Most of these failing students actively participate in class and exert substantial effort, but nevertheless receive low grades on exams. When surveyed about the resources that they find most helpful in learning the material, students cite practice exams with video solutions as the highest-rated component of the course. Although considerable research shows that practice exams can be an effective form of studying, the degree of effectiveness depends on how the students engage with the exams. Indeed, simply reviewing problems and solutions develops familiarity with the material to be tested, but it does not often support the needed problem-solving ability. Consequently, lower-performing students become overconfident in their problem-solving ability and believe they are prepared to do well on exams, but instead they receive failing grades. This project will investigate whether changing the format of students' interactions with the practice exams, to encourage earlier use and to provide a more accurate assessment of understanding, will result in more effective studying behavior. If so, it may enable students to make a better self-assessment of their readiness and increase performance on exams.This project aims to advance knowledge in three areas that are critical for improving the effectiveness of practice exams as a learning resource, especially for underperforming students. First, it will explore students' difficulty in assessing their level of understanding, also known as the Dunning Kroeger effect, in STEM gateway courses. Although this effect has been verified in a variety of situations, the investigators will not only measure the size of the effect but also develop insights about whether improving students' ability to assess their level of understanding improves their performance (i.e., Is the correlation causal?). Second, the project will explore the role that implementation has on how students interact with practice exams and their effectiveness in helping students learn. In particular, the investigators will attempt to measure the impact that providing students a more realistic score on their practice exam has on the accuracy of their own understanding of how well they know the material. Third, the project will explore the impact that encouraging distributed practice can have on student performance, as well as the effectiveness of a relatively simple approach to encouraging that practice. In addition to guiding future education research, the knowledge gained from this project should maximize the effectiveness of practice exams for the at-risk students who need them most. At the University of Illinois alone, the results of the research should improve learning for thousands of students every year who hope to become engineers. However, the results should transfer to introductory physics courses at other institutions and introductory courses in other disciplines, thus magnifying the potential impact of this work. This collaborative project involves investigators at the University of Illinois at Urbana-Champaign (Award DUE-2021099) and Purdue University (Award DUE-2021190). This project is funded by the Improving Undergraduate STEM Education (IUSE: EHR) program, which supports research and development projects to improve the effectiveness of STEM education for all students. Through the Engaged Student Learning track, the program supports the creation, exploration, and implementation of promising practices and tools.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目旨在通过改善学生在入门STEM课程中的成功来满足国家利益。对于许多大学和大学生，“门户”课程（例如介绍性物理学）为进一步学习的障碍提供了障碍。每个学期约有4,000名学生在伊利诺伊大学的物理系学习介绍课程。大多数学生都是工程专业，因为物理是工程学院所有课程所需的课程。尽管讲师使用了基于证据的教学方法，例如在线预设，主动学习讲座，“即时教学”，同伴教学以及在朗诵课程中解决协作问题，但近15％的学生没有通过入门课程。这些失败的学生中的大多数都积极参加课程并付出了巨大的努力，但是考试的成绩较低。当对他们发现最有帮助的材料的资源进行调查时，学生将视频解决方案作为课程最高的组成部分引用练习考试。尽管大量的研究表明，实践考试可以是一种有效的研究形式，但有效性程度取决于学生如何与考试互动。实际上，简单地审查问题和解决方案会熟悉要测试的材料，但通常不支持所需的解决问题能力。因此，表现较低的学生在解决问题的能力方面过分自信，并认为他们准备在考试中表现出色，但是他们获得了不及格的成绩。该项目将调查是否改变学生与实践考试的互动格式，以鼓励早期使用并提供对理解的更准确评估，这将导致更有效的研究行为。 如果是这样，它可能使学生能够更好地自我评估，并提高考试的表现。该项目旨在提高三个领域的知识，这对于提高实践考试作为学习资源的有效性至关重要，尤其是对于表现不佳的学生。首先，它将探索学生在STEM Gateway课程中评估他们的理解水平（也称为Dunning Kroeger效应）的困难。尽管在各种情况下已经验证了这种效果，但研究人员不仅会衡量效果的大小，而且还会对提高学生评估他们的理解水平的能力的见解有所提高（即，相关因果关系是否有效？）。其次，该项目将探讨实施在学生如何与实践考试互动及其在帮助学生学习方面的有效性方面的作用。特别是，调查人员将试图衡量为学生提供对练习考试的更现实得分对自己对材料了解如何的准确性的影响。第三，该项目将探讨鼓励分布式实践可以对学生绩效产生的影响，以及一种相对简单的方法来鼓励这种做法的有效性。除了指导未来的教育研究外，从该项目中获得的知识也应最大程度地提高最需要他们的高危学生的实践考试的有效性。仅在伊利诺伊大学，研究结果应每年改善成千上万的学生希望成为工程师的学习。但是，结果应转移到其他机构的介绍性物理课程以及其他学科的入门课程，从而放大了这项工作的潜在影响。 该合作项目涉及伊利诺伊大学Urbana-Champaign大学（授予2021099奖）和普渡大学（Dududue-2021190奖）的调查人员。 该项目由改善本科STEM教育（IUSE：EHR）计划提供资金，该计划支持研发项目，以提高所有学生的STEM教育的有效性。通过参与的学生学习轨道，该计划支持有希望的实践和工具的创建，探索和实施。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的知识分子优点和更广泛的影响评估的评估来支持的。

项目成果

Impact of more realistic and earlier practice exams on student metacognition, study behaviors, and exam performance

更现实和更早的模拟考试对学生元认知、学习行为和考试表现的影响

• DOI: 10.1103/physrevphyseducres.19.010130
• 发表时间: 2023
• 期刊: Physical Review Physics Education Research
• 影响因子: 3.1
• 作者: Zhang, Muxin;Morphew, Jason;Stelzer, Tim
• 通讯作者: Stelzer, Tim

Effect of online practice exams on student performance

在线模拟考试对学生成绩的影响

• DOI: 10.1119/perc.2019.pr.zhang
• 发表时间: 2020
• 期刊: American Association of Physics Teachers
• 作者: Zhang, Muxin;Engel, Andrew;Stelzer, Timothy;Morphew, Jason W.
• 通讯作者: Morphew, Jason W.

Learning is Quick: Evidence for the Impact of Learning Beliefs on Self-Regulated Learning

学习很快：学习信念对自我调节学习影响的证据

• 发表时间: 2023
• 期刊: American Educational Research Association
• 作者: Morphew, Jason;Shafer, Devyn;Zhang, Muxin;Girotti-Hernandez, Daniela;Stelzer, Tim
• 通讯作者: Stelzer, Tim

Case study on engineering design intervention in physics laboratories

物理实验室工程设计干预案例研究

• 发表时间: 2022
• 期刊: 2022 ASEE Annual Conference & Exposition Proceedings
• 作者: Kaufman-Ortiz, K.;Morphew, J. W.;Rebello, N. S.;& Rebello, C. M.
• 通讯作者: & Rebello, C. M.

Integrated STEM: Impact of Engineering Design and Computer Science in STEM Labs

综合 STEM：工程设计和计算机科学对 STEM 实验室的影响

• DOI: 10.5703/1288284317601
• 发表时间: 2023
• 期刊: Proceedings of the Eighth Annual STEM Education Conference
• 作者: Morphew, Jason;Lopez, Ruben;Bralin, Amir;Subramaniam, Ravishankar;Rebello, Sanjay;Rebello, Carina
• 通讯作者: Rebello, Carina