Collaborative Research: Using Artificial Intelligence to Transform Online Video Lectures into Effective and Inclusive Agent-Based Presentations

协作研究：利用人工智能将在线视频讲座转变为有效且包容的基于代理的演示

• 批准号: 2201019
• 负责人: Nicoletta Adamo-Villani
• 金额: $ 54.67万
• 依托单位: Purdue University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-05-01 至 2025-04-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2201019&HistoricalAwards=false
• 关键词: Collaborative; Research; Using; Artificial; Intelligence

Students in introductory STEM courses frequently encounter video lectures that include an instructor standing next to a progression of slides. A challenge with these video lectures is that students may lose interest in science when they see the instructors as unhelpful (e.g., when they portray negative emotions while teaching) or unwelcoming (e.g., when they do not reflect the gender, ethnic, and racial diversity of the audience). This project aims to develop, validate, study, and make publicly available an artificial-intelligence (AI)-based framework that takes existing online instructional video lectures on introductory STEM topics created by human instructors and transforms them into instructionally effective and inclusive agent-based presentations. This work is intended to help improve science instruction and attract students from under-represented groups.The research team will apply artificial intelligence methods to extract gesture and voice from instructional videos with human instructors and transform them into instruction delivered by a diverse set of emotionally and socially sensitive onscreen animated embodied agents. Experimental research studies will investigate: 1) whether students learn better from an AI-transformed version of a video lecture than the original instructor-made video lecture; and 2) which features of onscreen agents in AI-transformed video lectures produce improved learning outcomes and processes, comparing delivery from an onscreen agent who does or does not match the student's gender, ethnicity, or race. Overall, results from the project will impact access to high-quality inclusive STEM learning experiences.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

学习 STEM 入门课程的学生经常会遇到视频讲座，其中讲师站在一系列幻灯片旁边。这些视频讲座的一个挑战是，当学生认为教师无益（例如，当他们在教学时描绘负面情绪时）或不受欢迎（例如，当他们没有反映性别、民族和种族多样性时），他们可能会失去对科学的兴趣观众）。该项目旨在开发、验证、研究并公开一个基于人工智能 (AI) 的框架，该框架采用由人类教师创建的有关 STEM 入门主题的现有在线教学视频讲座，并将其转化为教学有效且具有包容性的基于代理的框架演示文稿。这项工作旨在帮助改善科学教学并吸引来自弱势群体的学生。研究团队将应用人工智能方法从人类讲师的教学视频中提取手势和声音，并将其转化为由不同的情感和情感群体提供的教学。具有社会敏感性的银幕动画化身。实验研究将调查：1）学生从人工智能转换的视频讲座版本中学得是否比原来的教师制作的视频讲座更好； 2) 通过比较与学生性别、民族或种族匹配或不匹配的屏幕代理的授课内容，人工智能改造的视频讲座中屏幕代理的哪些功能可以改善学习成果和过程。总体而言，该项目的结果将影响获得高质量的包容性 STEM 学习体验。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Recognizing and Relating to the Race/Ethnicity and Gender of Animated Pedagogical Agents

认识并联系动画教学人员的种族/民族和性别

• DOI: 10.1177/07356331231213932
• 发表时间: 2023-11
• 期刊: Journal of Educational Computing Research
• 影响因子: 4.8
• 作者: Zhao, Fangzheng;Mayer, Richard E.;Adamo;Mousas, Christos;Choi, Minsoo;Lam, Luchcha;Mukanova, Magzhan;Hauser, Klay
• 通讯作者: Hauser, Klay

Animated Pedagogical Agents Performing Affective Gestures Extracted from the GEMEP Dataset: Can People Recognize Their Emotions?

动画教学代理执行从 GEMEP 数据集中提取的情感手势：人们能识别自己的情绪吗？

• 发表时间: 2024-03
• 期刊: Social Informatics and Telecommunications Engineering
• 作者: Magzhan Mukanova; Nicoletta Adamo
• 通讯作者: Nicoletta Adamo

Developing a Scale for Measuring the Believability of Virtual Agents

开发衡量虚拟代理可信度的量表

• 发表时间: 2023-12
• 期刊: International Conference on Artificial Reality and Telexistence & Eurographics Symposium on Virtual Environments (ICAT-EGVE
• 作者: Guo; S.
• 通讯作者: S.

Effects of Body Type and Voice Pitch on Perceived Audio-Visual Correspondence and Believability of Virtual Characters

体型和音高对虚拟角色感知视听对应性和可信度的影响

• DOI: 10.1145/3605495.3605791
• 发表时间: 2023-08
• 期刊: ACM Symposium on Applied Perception
• 作者: Lam, Luchcha;Choi, Minsoo;Mukanova, Magzhan;Hauser, Klay;Zhao, Fangzheng;Mayer, Richard;Mousas, Christos;Adamo
• 通讯作者: Adamo