Collaborative Research: FW-HTF-RM: Artificial Intelligence Technology for Future Music Performers

合作研究：FW-HTF-RM：未来音乐表演者的人工智能技术

• 批准号: 2326198
• 负责人: Yeon Ji Yun
• 金额: $ 90万
• 依托单位: Purdue University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-10-01 至 2026-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2326198&HistoricalAwards=false
• 关键词: Collaborative; Research; FW; HTF; RM; Collaborative; Research; FW; HTF; RM

Nearly 300,000 people in the U.S are working in the music field. Recent progress in Artificial Intelligence (AI) has had profound impacts for music creators but has not yet had much impact on the practices of professional music performers. This project will investigate how AI technology could transform the work of future music performers for both individual practice (through developing coaching tools that analyze music performance) and collaborative practice (through developing systems that can substitute for missing performers in a group). The team will assess the tools in terms of two main questions: (1) When can AI technology provide measurable benefits to professional musicians' practice and performance? (2) What factors would affect future musicians' acceptance of AI technology in their work? The initial tools will focus on stringed instruments, but the underlying technologies are likely to be adaptable to a wide variety of performance contexts in the art and entertainment industry. The project team will also explore ways to use the tools to benefit students from disadvantaged groups; the tools may improve learning opportunities for students with limited access to music instruction. The team will recruit student researchers from groups underrepresented in STEM.This project will develop and integrate techniques from computer vision, natural language processing, and audio analysis to create two AI-enabled tools to support string music performers. The first tool, the Evaluator, aims to improve individual practice and performance. It analyzes a musician's sound and compares it to digitized music scores to detect deviations in intonation, rhythm, and dynamics. The Evaluator also analyzes captured video and compares it to a database of sample performers recorded with correct postures, allowing it to recommend better postures, which can both improve musical performance and reduce injury risks. The second tool, the Companion, aims to support common use cases when one or more musicians are missing from a group performance rehearsal. The Companion can play the part of one or several instruments to replace absent musicians, matching tempo, and style of the human musicians through audio analysis of their performance while also responding in real-time to verbal instructions. These tools will be developed and evaluated through a series of user studies, surveys, focus groups, and longitudinal deployments.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

美国近30万人正在音乐领域工作。人工智能（AI）的最新进展对音乐创建者产生了深远的影响，但尚未对专业音乐表演者的实践产生太大影响。该项目将调查AI技术如何通过个人实践（通过开发分析音乐表现的教练工具）和协作实践（通过开发可以代替小组中失踪表演者的系统）来改变未来音乐表演者的工作。团队将根据两个主要问题评估工具：（1）AI技术何时可以为专业音乐家的实践和表现提供可衡量的好处？ （2）哪些因素会影响未来的音乐家在工作中对AI技术的接受？最初的工具将集中在弦乐器上，但是基础技术可能适应艺术和娱乐行业的各种性能环境。项目团队还将探索使用这些工具来使弱势群体的学生受益的方法；这些工具可能会改善获得音乐教学障碍有限的学生的学习机会。该团队将从STEM中代表性不足的小组中招募学生研究人员。该项目将从计算机视觉，自然语言处理和音频分析中开发和整合技术，以创建两个支持弦乐音乐表演者的AI-abele工具。第一个工具是评估者，旨在改善个人实践和绩效。它分析了音乐家的声音，并将其与数字化的音乐分数进行比较，以检测语调，节奏和动态的偏差。评估者还分析了捕获的视频，并将其与以正确姿势录制的样本表演者的数据库进行了比较，从而可以推荐更好的姿势，从而可以改善音乐表现并降低伤害风险。第二个工具，即同伴，旨在在小组表演排练中缺少一个或多个音乐家时支持常见用例。 同伴可以通过对表演的音频分析来替换一种或几种乐器的角色，以取代缺失的音乐家，匹配的节奏和人类音乐家的风格，同时还可以实时回应口头说明。这些工具将通过一系列用户研究，调查，焦点小组和纵向部署进行开发和评估。该奖项反映了NSF的法定任务，并使用基金会的知识分子优点和更广泛的影响审查标准，认为值得通过评估来获得支持。