EAGER: Collaborative Research: Production of Second Language Speech: Formulation of Objective Speech Intelligibility Measures and Learner-specific Feedback

EAGER：协作研究：第二语言语音的产生：客观语音清晰度测量和学习者特定反馈的制定

• 批准号: 2140469
• 负责人: Okim Kang
• 金额: $ 12.3万
• 依托单位: Northern Arizona University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-10-01 至 2024-05-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2140469&HistoricalAwards=false
• 关键词: EAGER; Collaborative; Research; Production; Second

This Early-concept Grants for Exploratory Research (EAGER) project focuses on exploring and developing a novel operational collection of speech, language and perception-based measures to objectively assess speech intelligibility for second language (L2) speech production, as well as providing effective learner-specific feedback. With the rise of English as an international language, intelligibility-based successful communication has been emphasized over native-like accents. However, L2 teachers often raise concerns about learners’ slow or stagnant pronunciation progress. Several primary reasons for this problem may include difficulties in perceptually discerning changes in learners’ speech and interpreting learners’ speech patterns without any learner-specific intelligibility assessment profile. Today, teachers have no systematic way to assess each student’s speech changes, nor can students monitor and track feedback related to their pronunciation learning progression. Therefore, an exploratory and transformative method is introduced for measuring speech intelligibility that provides both teachers and learners with objective and individualized feedback. This exploratory project is proposed for EAGER funding in order to establish a baseline working framework for operational objective measure creation, and proof-of-concept assessment feedback for teachers and learners. This approach will help teachers gauge learners’ intelligibility levels and allow learners to self-regulate their learning progress incrementally over time. The long-term innovation is expected to benefit skilled US professionals from non-English speaking countries, who work in various STEM (science, technology, engineering, and mathematics) fields. Additionally, this interdisciplinary project provides various opportunities for hands-on training and experience for both graduate and undergraduate students in the fields of language education, applied linguistics, computer engineering, and speech technology.This project explores an idea to assess intelligibility in speech communications based on multiple individual speech measures for non-native speakers. The ideas are currently in their very early stages of development, and a large portion of the research ideas are untested. In order to establish the ground truth of potential individual speech production intelligibility measures, the implementation and feasibility of this intelligibility feedback approach must be validated with evidence. By employing advanced Automatic Speech Recognition-based accent classification technology based on machine learning, the team of researchers plan to provide learners with measured speech property information through operational and a discriminating set of objective speech intelligibility measures. The current innovation builds on language skill acquisition theory with a functional analytic-linguistic approach, arguing that explicit and metalinguistic feedback plays a pivotal role in moving learners forward in their L2 development. The vision is enabled by on-going research on auditory-based neurogram and spectrogram orthogonal polynomial measures that predict speech intelligibility, employing the learners’ unconstrained speech utterances. The project will contribute to the scientific knowledge of what constitutes L2 intelligible speech, understanding how individualized objective speech intelligibility feedback affects L2 speech development, and creating a foundational collection of speech/auditory/signal processing measures as well as ASR/DNN driven measures that assess a speaker’s intelligibility and identify efficient ways of implementing this technology in L2 learning contexts.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

这个早期概念探索性研究资助 (EAGER) 项目的重点是探索和开发基于语音、语言和感知的测量的新颖操作集合，以客观评估第二语言 (L2) 语音生成的语音清晰度，并为学习者提供有效的帮助- 具体反馈。随着英语作为一种国际语言的兴起，基于可理解性的成功沟通已经超过了母语口音，但是，第二语言教师经常对学习者的发音进度缓慢或停滞表示担忧。造成这个问题的几个主要原因可能包括在没有任何特定于学习者的清晰度评估配置文件的情况下难以感知地辨别学习者言语的变化和解释学习者的言语模式。如今，教师没有系统的方法来评估每个学生的言语变化，学生也无法监控。因此，我们引入了一种探索性和变革性的方法来测量语音清晰度，为教师和学习者提供客观和个性化的反馈。资金，以建立用于操作目标测量创建的基线工作框架，以及为教师和学习者提供概念验证评估反馈。这种方法将帮助教师衡量学习者的可懂度水平，并允许学习者逐步自我调节他们的学习进度。这项长期创新预计将使来自非英语国家、在各个 STEM（科学、技术、工程和数学）领域工作的美国专业人士受益。此外，这个跨学科项目还提供了各种实践机会。研究生和本科生的培训和经验语言教育、应用语言学、计算机工程和语音技术领域的学生。该项目探索了一种基于非母语人士的多种个人语音测量来评估语音交流清晰度的想法。这些想法目前还处于早期阶段。为了确定潜在的个人语音清晰度测量的基本事实，必须通过采用先进的自动语音来验证这种清晰度反馈方法的实施和可行性。基于机器学习的基于识别的口音分类技术，研究团队计划通过一组可操作的、有区别的客观语音清晰度测量，为学习者提供测量的语音属性信息。当前的创新建立在语言技能习得理论和功能分析的基础上。语言学方法，认为显式和元语言反馈在推动学习者的第二语言发展方面发挥着关键作用，这一愿景是通过对基于听觉的神经图和预测语音的频谱图正交多项式测量的持续研究而实现的。该项目将有助于加深对 L2 可理解语音构成的科学认识，了解个性化的客观语音可懂度反馈如何影响 L2 语音发展，并创建语音/听觉/信号处理措施的基础集合。以及 ASR/DNN 驱动的措施，用于评估说话者的清晰度并确定在 L2 学习环境中实施该技术的有效方法。该奖项反映了 NSF 的法定使命和通过使用基金会的智力优点和更广泛的影响审查标准进行评估，该项目被认为值得支持。

项目成果

Mobile-assisted pronunciation training with limited English proficiency: Learner background and technology attitude.

英语水平有限的移动辅助发音训练：学习者背景与技术态度

• 发表时间: 2022
• 期刊: Proceedings of the annual Pronunciation in Second Language Learning and Teaching Conference
• 作者: Hirschi, Kevin;Kang, Okim;Cucchiarini, Catia;Hansen, John;Strik, Helmer
• 通讯作者: Strik, Helmer