Research on Automated Formative Feedback of Problem-Solving Strategy Writing in Introductory Physics using Natural Language Processing

利用自然语言处理的物理导论中解题策略写作的自动形成反馈研究

• 批准号: 2300645
• 负责人: N. Sanjay Rebello
• 金额: $ 49.98万
• 依托单位: Purdue University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-07-01 至 2026-06-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2300645&HistoricalAwards=false
• 关键词: Research; Automated; Formative; Feedback; Problem

Problem solving is an essential 21st century STEM workforce skill, but in most undergraduate introductory STEM courses students use ineffective problem-solving strategies. The pedagogical challenge is that these courses typically have very large enrollments which prohibits instructors' providing of real-time feedback to students on their unproductive strategies, given how labor intensive providing such individualized feedback is. The investigators propose to use natural language processing (NLP) to develop an automated system that will provide personalized formative feedback to students in undergraduate physics courses about their problem-solving strategies. There are two research questions: (1) How accurately can an NLP classifier score such essays and how well can it generalize across types of problems?, and (2) How well does providing students with such feedback improve their problem-solving strategies and actual ability to solve problems? This project leverages existing research in STEM education that has shown that the use of strategy writing, and real-time formative assessment can improve students' problem-solving skills. Starting with a vast corpus of existing data, the investigators will use state-of-the-art supervised and unsupervised learning to train a machine learning algorithm (MLA) to classify strategies articulated by students. Their study will span three phases with increasing generalization, first generalizing over vocabulary used by students to describe their strategies for solving specific problems, then generalizing over sets of isomorphic problems using the same strategies, and finally generalizing over classes of transfer problems that have the same underlying principles but different surface features. Their goal is to determine the accuracy with which an MLA can be trained to provide feedback on short essays written by students describing their strategies for solving a problem. They will also investigate the extent to which real-time formative feedback to students, based on their own strategy essays, can help them to refine their strategies iteratively in order to solve problems with increasing generality. In developing and training their machine learning algorithm, the researchers will implement proven strategies to address issues of fairness in artificial intelligence (AI). The ultimate goal of the project is to transform the development of expert-like problem-solving strategies in STEM undergraduates and thereby potentially to increase their retention in STEM majors.This project is supported by NSF's EDU Core Research (ECR) program. The ECR program emphasizes fundamental STEM education research that generates foundational knowledge in the field. Investments are made in critical areas that are essential, broad, and enduring: STEM learning and STEM learning environments, broadening participation in STEM, and STEM workforce development.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

解决问题是21世纪必不可少的STEM劳动力技能，但是在大多数本科入门的STEM课程中，学生使用无效的解决问题的策略。教学挑战是，这些课程通常有很大的入学人数，这些入学人数禁止教师向学生提供实时反馈，鉴于劳动密集型提供了这种个性化的反馈意见，因此向学生提供了无效的策略。调查人员建议使用自然语言处理（NLP）开发自动化系统，该系统将为本科物理学课程的学生提供有关解决问题的策略的个性化形成性反馈。有两个研究问题：（1）NLP分类器对这样的论文得分的准确程度如何，以及它可以跨越类型的问题概括？以及（2）为学生提供此类反馈的能力如何改善解决问题的策略和解决问题的实际能力？该项目利用STEM教育中的现有研究表明，使用策略写作和实时形成性评估可以提高学生解决问题的技能。从大量的现有数据开始，研究人员将使用最先进的监督和无监督的学习来培训机器学习算法（MLA），以对学生阐明的策略进行分类。他们的研究将跨越三个阶段，首先将学生用于描述他们解决特定问题的策略，然后使用相同的策略概括了一组同构问题，最后将具有相同基本原理但不同表面特征的转移问题概括为概括。他们的目标是确定可以培训MLA的准确性，以提供有关学生描述解决问题策略的简短文章的反馈。他们还将根据自己的策略论文来研究对学生的实时形成性反馈的程度，可以帮助他们迭代地完善策略，以解决一般性的问题。在开发和培训其机器学习算法时，研究人员将实施验证的策略，以解决人工智能公平性问题（AI）。该项目的最终目标是改变STEM本科生中类似专家的问题解决策略的发展，从而有可能增加其在STEM专业的保留。该项目得到了NSF的EDU核心研究（ECR）计划的支持。 ECR计划强调了基本的STEM教育研究，该研究在该领域产生了基础知识。投资是在重要，广泛且持久的关键领域进行的：STEM学习和STEM学习环境，扩大参与STEM和STEM劳动力发展。该奖项反映了NSF的法定任务，并被认为值得通过基金会的知识分子优点和更广泛的影响来通过评估来进行评估。