Collaborative Research: Advancing STEM Online Learning by Augmenting Accessibility with Explanatory Captions and AI

协作研究：通过解释性字幕和人工智能增强可访问性，推进 STEM 在线学习

• 批准号: 2118824
• 负责人: Qi Wang
• 金额: $ 13.39万
• 依托单位: Gallaudet University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-09-15 至 2024-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2118824&HistoricalAwards=false
• 关键词: Collaborative; Research; Advancing; STEM; Online; Collaborative; Research; Advancing; STEM; Online

Videos are a popular medium for online learning, in which captions are essential for increasing accessibility to students for effective learning. This research identifies two types of video captions: typical closed captions and explanatory captions. Closed captions are a text representation of the spoken part of a video. Explanatory captions are created to give students insights into the visual, textual, and audio content of a video. Existing technologies have focused on automatically generating or improving the quality of closed captions. For STEM learning, explanatory captions have the potential to play a new role in learning. This project will work to devise effective Q/A mechanisms and effective interaction designs that enable students and instructors to generate explanatory captions for STEM videos in a collaborative manner. The proposed technologies will augment accessibility and learning experiences for under-served populations, including the Deaf and Hard-of-Hearing (DHH) community, made up of 48 million Americans, while also improving comprehension for non-native English speakers, even those without hearing impairments. Evaluation sites include both Gallaudet University, the world’s only liberal arts university dedicated exclusively to educating DHH learners, and the University of Illinois at Urbana-Champaign, which has the largest international student population amongst U.S. public institutions and supports students with disabilities in inclusive learning environments. This interdisciplinary research draws from and contributes to both computer science and learning science, and accessibility practices in the following areas. The first step is discovering new knowledge about how accessibility-enabled videos (with explanatory and closed captions) broaden the participation of under-served populations in STEM learning. This will provide the foundation for developing a theory of how explanatory captions can contribute to learning and effective mechanisms, based on crowdsourced human contributions and machine learning algorithms, to create these explanatory captions for STEM videos at different learning stages (e.g., preparing, tracking, trouble-shooting, and reflecting). The investigators will then use the theory to create a novel chatbot that enables knowledge sharing for students with diverse backgrounds. Theoretical frameworks--ICAP (interactive, constructive, active, and passive) and Community of Inquiry will guide the evaluation of how explanatory captions and chatbots can contribute to learning. Finally, the team will acquire empirical understanding of how augmented accessibility with AI agents (e.g., chatbots) impacts students' and instructors’ practices.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

视频是在线学习的流行媒介，其中标题对于增加学生的有效学习至关重要。这项研究确定了两种类型的视频标题：典型的封闭字幕和利用字幕。封闭的标题是视频口语部分的文字表示。创建剥削字幕是为了使学生了解视频的视觉，文字和音频内容。现有技术的重点是自动生成或改善封闭标题的质量。对于STEM学习，爆炸字幕有可能在学习中发挥新作用。该项目将致力于设计有效的Q/A机制和有效的互动设计，使学生和讲师能够以协作方式为STEM视频产生爆炸性字幕。拟议的技术将增强服务不足的人群的可及性和学习经验，包括聋哑人和障碍（DHH）社区，由4,800万美国人组成，同时也提高了非本地人英语说话者的理解，甚至没有听力障碍的人。评估网站包括Gallaudet University，全球唯一专门致力于教育DHH学习者的文科大学和Urbana-Champaign大学的伊利诺伊大学，该大学在美国公共机构中拥有最大的国际学生人口，并在包容性学习环境中为残疾学生提供支持。这项跨学科研究借鉴了计算机科学和学习科学以及以下领域的可及性实践的贡献。第一步是发现有关支持可及性的视频（带有爆炸性和封闭式字幕）如何扩大服务不足人群在STEM学习中的参与的新知识。这将为开发一个理论，说明爆炸性标题如何根据人类的贡献和机器学习算法为学习和有效机制做出贡献，以在不同的学习阶段为STEM视频创建这些利用标题（例如，准备，跟踪，跟踪，故障障碍和反射）。然后，调查人员将使用该理论创建一种新颖的聊天机器人，该聊天机器人可以为有潜水员背景的学生共享知识。理论框架 - ICAP（交互式，建设性，主动和被动）和询问社区将指导评估利用字幕和聊天机器人如何有助于学习。最后，该团队将对AI代理商（例如聊天机器人）如何增强可访问性影响学生和讲师的实践的实证理解。该奖项反映了NSF的法定任务，并被认为是通过基金会的知识分子优点和更广泛的影响审查标准通过评估来评估的。