Collaborative Research: STEM Learning Embedded in a Machine-in-the-Loop Collaborative Story Writing Game

协作研究：嵌入机器在环协作故事写作游戏中的 STEM 学习

• 批准号: 2202506
• 负责人: Mohit Iyyer
• 金额: $ 62.14万
• 依托单位: University of Massachusetts Amherst
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-09-01 至 2025-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2202506&HistoricalAwards=false
• 关键词: Collaborative; Research; STEM; Learning; Embedded

Developing “21st century skills” such as collaboration, communication, critical thinking, and creativity (the 4Cs) has become increasingly important for students to keep up with the ever-evolving labor market of the future. Teaching the 4Cs effectively and efficiently requires deeply intertwining them with core content knowledge areas, since the acquisition of domain knowledge can bolster students’ development of these soft skills. In this project, the investigators take a step towards combining 4C skill development with STEM education by developing a collaborative writing game in which multiple students work together to craft a narrative around embedded STEM education elements. As a key innovation, the investigators will embed this collaborative writing game with natural language processing and artificial intelligence (AI)-based tools to automate fact-checking, feedback, knowledge tracing, and narrative story arc suggestions, which will facilitate students’ progress toward mastery while reducing teacher workload. Overall, this project has the potential to increase student engagement in STEM learning activities and improve learning outcomes. The project will be grounded in StoriumEdu, a collaborative story writing platform, therefore directly benefiting its user base of 2,000 K-12 classrooms with over 27,000 students and potentially an even larger number of students through the dissemination of the team’s research findings. This major technical goals of this project are intended to augment scientific writing instruction with AI-based tools. To achieve these goals, the project will develop novel technologies that automatically provide writing assistance and feedback, and these tools will be deployed into K-12 classrooms via the StoriumEdu platform in order to evaluate their effectiveness. A core technical challenge is to assess the factuality of student writing by building machine learning models for fact-checking. The team proposes to design retrieval-augmented neural networks that can localize spans within student-written text that exhibit scientific misunderstandings. These spans will then be connected with relevant passages from textbooks or online articles to enable students to easily correct their errors. After developing fact-checking methods, the team will also focus on knowledge tracing, which allows measuring student progress over time in terms of which concepts they have mastered or are still struggling with. The knowledge tracing models will be developed with feedback from scientific literacy experts. The output of these models informs the final aspect of this project, which aims to generate narrative progressions associated with conceptual misunderstandings. This will allow students to engage more strongly with concepts that they have yet to master, which maximizes the writing platform’s pedagogical potential. Taken as a whole, this project’s research contributions synthesize novel NLP methods with educational progress tracking and feedback systems in an effort to improve STEM learning.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

开发“ 21世纪的技能”，例如协作，沟通，批判性思维和创造性（4C），对于学生来说，跟上不断发展的未来劳动力市场变得越来越重要。有效，有效地教授4C需要将它们与核心内容知识领域的纠结深入，因为获得域知识可以增强学生对这些软技能的发展。在这个项目中，调查人员通过开发一个协作写作游戏迈出了将4C技能发展与STEM教育相结合的一步，其中多个学生共同努力，围绕嵌入式STEM教育元素制作叙事。作为一项关键创新，研究人员将使用自然语言处理和人工智能（AI）的工具嵌入此协作写作游戏，以自动化事实检查，反馈，知识追踪和叙事故事的建议，这将促进学生在减少教师工作的同时迈向精通学生的进步。总体而言，该项目有可能增加学生参与STEM学习活动并改善学习成果。该项目将基于协作故事写作平台Storiumedu，因此直接使其2,000 k-11的用户基础与27,000多名学生一起受益，并通过传播团队的研究结果，并有可能增加更多的学生。该项目的主要技术目标旨在使用基于AI的工具来增强科学写作说明。为了实现这些目标，该项目将开发新的技术，这些技术会自动提供写作帮助和反馈，并且这些工具将通过Storiumedu平台部署到K-12教室中，以评估其有效性。核心技术挑战是评估学生通过建立机器学习模型进行事实检查的事实。团队提出的建议设计检索的神经网络，该网络可以将跨度定位在学生写的文本中，这些文本存在科学的误解。然后，这些跨度将与从教科书或在线文章中的相关段落联系起来，以使学生可以轻松纠正错误。在开发了事实检查方法之后，团队还将专注于知识跟踪，这可以在他们掌握或仍在努力的哪些概念方面衡量学生的进步。知识追踪模型将通过科学素养专家的反馈来开发。这些模型的输出为该项目的最后一个方面提供了信息，该方面旨在产生与概念上的误解相关的叙事进程。这将使学生能够更加强烈地参与尚未掌握的概念，从而最大程度地发挥了写作平台的教学潜力。从总体上讲，该项目的研究贡献将新颖的NLP方法与教育进度跟踪和反馈系统合成，以改善STEM学习。该奖项反映了NSF的法定任务，并被认为是通过基金会的智力优点和更广泛的影响来通过评估来获得支持的支持。

项目成果

LongEval: Guidelines for Human Evaluation of Faithfulness in Long-form Summarization

• DOI: 10.48550/arxiv.2301.13298
• 发表时间: 2023-01
• 期刊: ArXiv
• 作者: Kalpesh Krishna;Erin Bransom;Bailey Kuehl;Mohit Iyyer;Pradeep Dasigi;Arman Cohan;Kyle Lo

Open-ended Knowledge Tracing for Computer Science Education

• DOI: 10.18653/v1/2022.emnlp-main.254
• 发表时间: 2022
• 作者: Naiming Liu;Zichao Wang

A Critical Evaluation of Evaluations for Long-form Question Answering

对长式问答评估的批判性评估

• DOI: 10.18653/v1/2023.acl-long.181
• 发表时间: 2023
• 期刊: Association for Computational Linguistics
• 作者: Xu, Fangyuan;Song, Yixiao;Iyyer, Mohit;Choi, Eunsol
• 通讯作者: Choi, Eunsol