Research on Automated Formative Feedback of Problem-Solving Strategy Writing in Introductory Physics using Natural Language Processing

利用自然语言处理的物理导论中解题策略写作的自动形成反馈研究

• 批准号: 2300645
• 负责人: N. Sanjay Rebello
• 金额: $ 49.98万
• 依托单位: Purdue University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-07-01 至 2026-06-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2300645&HistoricalAwards=false
• 关键词: Research; Automated; Formative; Feedback; Problem

Problem solving is an essential 21st century STEM workforce skill, but in most undergraduate introductory STEM courses students use ineffective problem-solving strategies. The pedagogical challenge is that these courses typically have very large enrollments which prohibits instructors' providing of real-time feedback to students on their unproductive strategies, given how labor intensive providing such individualized feedback is. The investigators propose to use natural language processing (NLP) to develop an automated system that will provide personalized formative feedback to students in undergraduate physics courses about their problem-solving strategies. There are two research questions: (1) How accurately can an NLP classifier score such essays and how well can it generalize across types of problems?, and (2) How well does providing students with such feedback improve their problem-solving strategies and actual ability to solve problems? This project leverages existing research in STEM education that has shown that the use of strategy writing, and real-time formative assessment can improve students' problem-solving skills. Starting with a vast corpus of existing data, the investigators will use state-of-the-art supervised and unsupervised learning to train a machine learning algorithm (MLA) to classify strategies articulated by students. Their study will span three phases with increasing generalization, first generalizing over vocabulary used by students to describe their strategies for solving specific problems, then generalizing over sets of isomorphic problems using the same strategies, and finally generalizing over classes of transfer problems that have the same underlying principles but different surface features. Their goal is to determine the accuracy with which an MLA can be trained to provide feedback on short essays written by students describing their strategies for solving a problem. They will also investigate the extent to which real-time formative feedback to students, based on their own strategy essays, can help them to refine their strategies iteratively in order to solve problems with increasing generality. In developing and training their machine learning algorithm, the researchers will implement proven strategies to address issues of fairness in artificial intelligence (AI). The ultimate goal of the project is to transform the development of expert-like problem-solving strategies in STEM undergraduates and thereby potentially to increase their retention in STEM majors.This project is supported by NSF's EDU Core Research (ECR) program. The ECR program emphasizes fundamental STEM education research that generates foundational knowledge in the field. Investments are made in critical areas that are essential, broad, and enduring: STEM learning and STEM learning environments, broadening participation in STEM, and STEM workforce development.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

解决问题是 21 世纪 STEM 劳动力的一项重要技能，但在大多数本科 STEM 入门课程中，学生使用的解决问题策略无效。教学上的挑战是，这些课程通常有非常大的注册人数，考虑到提供这种个性化反馈的劳动强度，教师无法向学生提供有关其非生产性策略的实时反馈。研究人员建议使用自然语言处理（NLP）来开发一个自动化系统，该系统将为本科物理课程的学生提供有关其解决问题策略的个性化形成性反馈。有两个研究问题：（1）NLP 分类器对此类论文进行评分的准确度如何，以及它对不同类型问题的概括能力如何？（2）为学生提供此类反馈能在多大程度上改善他们的问题解决策略和实际情况？解决问题的能力？该项目利用了 STEM 教育的现有研究，这些研究表明，使用策略写作和实时形成性评估可以提高学生解决问题的能力。研究人员将从大量现有数据开始，使用最先进的监督和无监督学习来训练机器学习算法（MLA），对学生表达的策略进行分类。他们的研究将跨越三个阶段，并不断增强泛化能力，首先泛化学生用来描述解决特定问题的策略的词汇，然后泛化使用相同策略的同构问题集，最后泛化具有相同策略的迁移问题类基本原理不同但表面特征不同。他们的目标是确定 MLA 训练的准确性，以便为学生撰写的描述解决问题策略的短文提供反馈。他们还将研究根据学生自己的策略论文向学生提供的实时形成性反馈在多大程度上可以帮助他们迭代地完善策略，以解决越来越普遍的问题。在开发和训练机器学习算法时，研究人员将实施经过验证的策略来解决人工智能 (AI) 的公平问题。该项目的最终目标是转变 STEM 本科生专家式问题解决策略的发展，从而有可能提高他们在 STEM 专业的保留率。该项目得到了 NSF 的 EDU 核心研究 (ECR) 计划的支持。 ECR 项目强调基础 STEM 教育研究，产生该领域的基础知识。投资针对重要、广泛和持久的关键领域：STEM 学习和 STEM 学习环境、扩大 STEM 参与以及 STEM 劳动力发展。该奖项反映了 NSF 的法定使命，并通过使用基金会的评估进行评估，被认为值得支持。智力价值和更广泛的影响审查标准。