Collaborative Research: Common Error Diagnostics and Support in Short-answer Math Questions

合作研究：简答数学问题中的常见错误诊断和支持

• 批准号: 2118725
• 负责人: Neil Heffernan
• 金额: $ 23.93万
• 依托单位: Worcester Polytechnic Institute
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2118725&HistoricalAwards=false
• 关键词: Collaborative; Research; Common; Error; Diagnostics

One important way to help struggling students improve in math is to deliver personalized support that addresses their specific weaknesses. Many math questions have common wrong answers (CWAs) that correspond to specific errors students make during their answering process, caused by misconceptions or a general lack of knowledge on certain math skills. To date, CWA identification and support remains a labor-intensive process at a limited scale because it requires significant effort by teachers and/or domain experts. In this project, the investigators will develop artificial intelligence (AI)-based mechanisms that can automatically identify CWAs from students’ answers to short-answer math questions and diagnose errors. Once these errors are identified, the investigators will enlist the help of teachers to design feedback and support mechanisms in various formats such as textual feedback messages and short videos. In turn, the investigators will integrate these diagnosis and effective support mechanisms into a teacher interface to support them in either classrooms or online learning environments. Overall, this project has the potential to lead to i) better understanding of CWAs in math questions and the underlying errors and ii) effective CWA support mechanisms for each error type. The project will be grounded in ASSISTments, a free web-based learning platform, therefore directly benefiting the 500,000 US students and 20,000 teachers using it and potentially an even larger number of students and teachers through the dissemination of research findings. This project consists of four main research activities. First, the investigators will leverage math expression embedding methods to learn the representations of student errors by clustering CWAs across multiple questions in the latent math expression embedding vector space. These learned representations will enable the automated diagnosis of student errors in real time. Second, the investigators will develop new knowledge tracing algorithms that go beyond typical correctness analysis and analyze the full answer each student submits to each question. These algorithms will enable the automated tracking of students’ progress in correcting their errors. Third, the investigators will crowdsource multiple types of student support from teachers and integrate both student error diagnostics and support mechanisms into the existing ASSISTments teacher interface. This interface will provide feedback to teachers on which students are struggling in real time and recommend a support, which the teacher can either adopt and customize or reject and create their own support instead. Fourth, the investigators will conduct a randomized controlled trial to evaluate the effectiveness of each support mechanism in helping students correct their errors. This experiment will identify which support mechanisms are most effective at helping students correct each error type and improving learning outcomes.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

帮助有困难的学生提高数学成绩的一个重要方法是提供个性化支持，解决他们的具体弱点。许多数学问题都有常见的错误答案 (CWA)，这些错误答案与学生在回答过程中因误解或普遍缺乏而犯的具体错误相对应。迄今为止，CWA 识别和支持仍然是一个规模有限的劳动密集型过程，因为它需要教师和/或领域专家的巨大努力。在该项目中，研究人员将开发人工智能 (AI)。基于可自动识别 CWA 的机制一旦发现这些错误，研究人员将寻求教师的帮助，设计各种形式的反馈和支持机制，例如文字反馈信息和短视频。研究人员将把这些诊断和有效的支持机制整合到教师界面中，以在课堂或在线学习环境中为他们提供支持。总体而言，该项目有可能导致 i) 更好地理解数学问题中的 CWA 和潜在错误，以及 ii)。针对每种错误类型的有效 CWA 支持机制。该项目将以 ASSISTments 为基础，这是一个免费的网络学习平台，因此使用该平台的 500,000 名美国学生和 20,000 名教师将直接受益，并可能通过研究成果的传播使更多的学生和教师受益。研究人员将利用数学表达式嵌入方法，通过在潜在数学表达式嵌入向量空间中的多个问题上对 CWA 进行聚类，来学习学生错误的表示。其次，研究人员将开发新的知识追踪算法，超越典型的正确性分析，并分析每个学生提交的每个问题的完整答案。第三，研究人员将从教师那里众包多种类型的学生支持，并将学生错误诊断和支持机制集成到现有的 ASSISTments 教师界面中。该界面将为教师提供学生实际遇到困难的反馈。时间并推荐一个支持，老师第四，研究人员将进行随机对照试验，以评估每种支持机制在帮助学生纠正错误方面的有效性。帮助学生纠正每种错误类型并提高学习成果。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Comparing Different Approaches to Generating Mathematics Explanations Using Large Language Models

比较使用大型语言模型生成数学解释的不同方法

• 发表时间: 2023-07
• 期刊: AIED 2023
• 作者: Prihar, Ethan;Lee, Morgan;Hopman, Mia;Kalai, Adam T.;Vempala, Sofia;Wang, Allison;Wickline, Gabriel;Murray, Aly;Heffernan, Neil
• 通讯作者: Heffernan, Neil

How Common are Common Wrong Answers? Crowdsourcing Remediation at Scale.

常见错误答案有多常见？

• 发表时间: 2023-07
• 期刊: Learning@Scale 2023
• 作者: Gurung, Ashish;Lee, Morgan;Baral, Sami;Sales, Adam;Vanacore, Kirk;McReynolds, Andrew;Kreisberg, Hilary;Heffernan, Cristina;Haim, Aaron;Heffernan, Neil
• 通讯作者: Heffernan, Neil

How to Open Science: A Principle and Reproducibility Review of the Learning Analytics and Knowledge Conference

如何开放科学：学习分析和知识会议的原理和可重复性回顾

• 发表时间: 2023-03
• 期刊: LAK ’23: International Conference on Learning Analytics & Knowledge
• 作者: Haim, Aaron;Shaw, Stacy;Heffernan, Neil
• 通讯作者: Heffernan, Neil

Assessing the ality of Large Language Models in Generating Mathematics Explanations

评估大型语言模型在生成数学解释中的能力

• 发表时间: 2023-07
• 期刊: Denmark.
• 作者: Wang, Allison;Prihar, Ethan;Heffernan, Neil
• 通讯作者: Heffernan, Neil

A Bandit You Can Trust

值得信赖的强盗

• 发表时间: 2023-07
• 期刊: UMAP 2023
• 作者: Prihar, Ethan;Sales, Adam;Heffernan, Neil
• 通讯作者: Heffernan, Neil