Collaborative Research: Advancing STEM Online Learning by Augmenting Accessibility with Explanatory Captions and AI

协作研究：通过解释性字幕和人工智能增强可访问性，推进 STEM 在线学习

• 批准号: 2118824
• 负责人: Qi Wang
• 金额: $ 13.39万
• 依托单位: Gallaudet University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-09-15 至 2024-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2118824&HistoricalAwards=false
• 关键词: Collaborative; Research; Advancing; STEM; Online

Videos are a popular medium for online learning, in which captions are essential for increasing accessibility to students for effective learning. This research identifies two types of video captions: typical closed captions and explanatory captions. Closed captions are a text representation of the spoken part of a video. Explanatory captions are created to give students insights into the visual, textual, and audio content of a video. Existing technologies have focused on automatically generating or improving the quality of closed captions. For STEM learning, explanatory captions have the potential to play a new role in learning. This project will work to devise effective Q/A mechanisms and effective interaction designs that enable students and instructors to generate explanatory captions for STEM videos in a collaborative manner. The proposed technologies will augment accessibility and learning experiences for under-served populations, including the Deaf and Hard-of-Hearing (DHH) community, made up of 48 million Americans, while also improving comprehension for non-native English speakers, even those without hearing impairments. Evaluation sites include both Gallaudet University, the world’s only liberal arts university dedicated exclusively to educating DHH learners, and the University of Illinois at Urbana-Champaign, which has the largest international student population amongst U.S. public institutions and supports students with disabilities in inclusive learning environments. This interdisciplinary research draws from and contributes to both computer science and learning science, and accessibility practices in the following areas. The first step is discovering new knowledge about how accessibility-enabled videos (with explanatory and closed captions) broaden the participation of under-served populations in STEM learning. This will provide the foundation for developing a theory of how explanatory captions can contribute to learning and effective mechanisms, based on crowdsourced human contributions and machine learning algorithms, to create these explanatory captions for STEM videos at different learning stages (e.g., preparing, tracking, trouble-shooting, and reflecting). The investigators will then use the theory to create a novel chatbot that enables knowledge sharing for students with diverse backgrounds. Theoretical frameworks--ICAP (interactive, constructive, active, and passive) and Community of Inquiry will guide the evaluation of how explanatory captions and chatbots can contribute to learning. Finally, the team will acquire empirical understanding of how augmented accessibility with AI agents (e.g., chatbots) impacts students' and instructors’ practices.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

视频是一种流行的在线学习媒介，其中字幕对于提高学生的有效学习能力至关重要。这项研究确定了两种类型的视频字幕：典型的隐藏式字幕和解释性字幕是口头部分的文本表示。创建解释性字幕是为了让学生深入了解视频的视觉、文本和音频内容。对于 STEM 学习，解释性字幕具有以下功能：自动生成或提高隐藏式字幕的质量。该项目将致力于设计有效的问答机制和有效的交互设计，使学生和教师能够以协作方式为 STEM 视频生成解释性字幕。所提出的技术将增强可访问性和学习体验。为服务不足的人群，包括由 4800 万美国人组成的聋哑和听力障碍 (DHH) 群体，同时还提高非英语母语人士的理解能力，即使是那些没有听力障碍的人。 评估网站包括加劳德特大学。 ,伊利诺伊大学厄巴纳-香槟分校是世界上唯一一所专门致力于教育 DHH 学习者的文科大学，该大学拥有美国公立机构中国际学生人数最多的大学，并在包容性学习环境中支持残疾学生。为计算机科学和学习科学以及以下领域的无障碍实践做出贡献 第一步是发现有关无障碍视频（带有解释性和隐藏式字幕）如何扩大服务不足人群对 STEM 学习的参与的新知识。这将为开发解释性字幕如何促进学习的理论和基于众包人类贡献和机器学习算法的有效机制奠定基础，为不同学习阶段的 STEM 视频创建这些解释性字幕（例如，准备、跟踪、然后，研究人员将使用该理论创建一个新颖的聊天机器人，为具有不同背景的学生提供知识共享——ICAP（交互式、建设性、主动和被动）和社区。调查将指导评估解释性字幕和聊天机器人如何促进学习。最后，团队将获得对人工智能代理（例如聊天机器人）的增强可访问性如何影响学生和教师实践的实证理解。该奖项反映了 NSF 的法定使命。通过使用基金会的智力优点和更广泛的影响审查标准进行评估，并被认为值得支持。