EAGER: Collaborative Research: Second Language Speech Production: Formulation of Objective Speech Intelligibility Measures and Learner-Specific Feedback

EAGER：协作研究：第二语言语音生成：客观语音清晰度测量和学习者特定反馈的制定

• 批准号: 2140414
• 负责人: Stephen Looney
• 金额: $ 4.2万
• 依托单位: Pennsylvania State Univ University Park
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-10-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2140414&HistoricalAwards=false
• 关键词: EAGER; Collaborative; Research; Second; Language

This Early-concept Grants for Exploratory Research (EAGER) project focuses on exploring and developing a novel operational collection of speech, language and perception-based measures to objectively assess speech intelligibility for second language (L2) speech production, as well as providing effective learner-specific feedback. With the rise of English as an international language, intelligibility-based successful communication has been emphasized over native-like accents. However, L2 teachers often raise concerns about learners’ slow or stagnant pronunciation progress. Several primary reasons for this problem may include difficulties in perceptually discerning changes in learners’ speech and interpreting learners’ speech patterns without any learner-specific intelligibility assessment profile. Today, teachers have no systematic way to assess each student’s speech changes, nor can students monitor and track feedback related to their pronunciation learning progression. Therefore, an exploratory and transformative method is introduced for measuring speech intelligibility that provides both teachers and learners with objective and individualized feedback. This exploratory project is proposed for EAGER funding in order to establish a baseline working framework for operational objective measure creation, and proof-of-concept assessment feedback for teachers and learners. This approach will help teachers gauge learners’ intelligibility levels and allow learners to self-regulate their learning progress incrementally over time. The long-term innovation is expected to benefit skilled US professionals from non-English speaking countries, who work in various STEM (science, technology, engineering, and mathematics) fields. Additionally, this interdisciplinary project provides various opportunities for hands-on training and experience for both graduate and undergraduate students in the fields of language education, applied linguistics, computer engineering, and speech technology.This project explores an idea to assess intelligibility in speech communications based on multiple individual speech measures for non-native speakers. The ideas are currently in their very early stages of development, and a large portion of the research ideas are untested. In order to establish the ground truth of potential individual speech production intelligibility measures, the implementation and feasibility of this intelligibility feedback approach must be validated with evidence. By employing advanced Automatic Speech Recognition-based accent classification technology based on machine learning, the team of researchers plan to provide learners with measured speech property information through operational and a discriminating set of objective speech intelligibility measures. The current innovation builds on language skill acquisition theory with a functional analytic-linguistic approach, arguing that explicit and metalinguistic feedback plays a pivotal role in moving learners forward in their L2 development. The vision is enabled by on-going research on auditory-based neurogram and spectrogram orthogonal polynomial measures that predict speech intelligibility, employing the learners’ unconstrained speech utterances. The project will contribute to the scientific knowledge of what constitutes L2 intelligible speech, understanding how individualized objective speech intelligibility feedback affects L2 speech development, and creating a foundational collection of speech/auditory/signal processing measures as well as ASR/DNN driven measures that assess a speaker’s intelligibility and identify efficient ways of implementing this technology in L2 learning contexts.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

这项早期概念授予探索性研究（急切）项目的重点是探索和开发基于语言和感知的措施的新型操作集合，以客观地评估第二语言的语音智力（L2）语音生产，并提供有效的学习者反馈。随着英语作为一种国际语言的兴起，基于情报的成功沟通被强调了类似于本人的口音。但是，L2老师经常对学习者的缓慢或停滞的发音进展引起人们的担忧。造成此问题的几个主要原因可能包括在没有任何特定于学习者的情报评估概况的情况下，在学习者的语音上刻张变化和解释学习者的语音模式的困难。如今，教师没有系统的方法来评估每个学生的语音变化，学生也不能监视和跟踪与他们的发音学习进步有关的反馈。因此，引入了一种探索性和变革性的方法，用于测量语音智能，从而为教师和学习者提供客观和个性化的反馈。该探索性项目是为了建立一个基线工作框架，以建立一个基线工作框架，以创建运营客观的测量框架，并为教师和学习者提供概念验证评估反馈。这种方法将帮助教师衡量学习者的清晰度水平，并使学习者随着时间的推移逐步自我调节学习进步。长期创新有望使来自非英语国家的熟练美国专业人士受益，这些国家从事各种STEM（科学，技术，工程和数学）领域的工作。此外，该跨学科项目为语言教育，应用语言学，计算机工程和语音技术领域的研究生和本科生的动手培训和经验提供了各种机会。该项目探讨了一种评估基于多个个人语音测量的语音通信中情报的想法。这些想法目前处于发展的早期阶段，并且很大一部分研究思想未经测试。为了建立潜在的个人语音生产智能措施的基础真理，必须用证据来验证这种清晰度反馈方法的实施和可行性。通过基于机器学习的高级自动语音识别分类技术，研究人员计划通过操作和一组歧视客观的语音智能指标为学习者提供测量的语音属性信息。当前的创新以功能分析语言的方法为基础，以语言技能的获取理论为基础，认为显式和金属语言反馈在使学习者在L2开发中前进时起着关键作用。通过对基于听觉的神经图和频谱谱图正交多项式测量的持续研究，可以预测语音清晰度，并采用学习者使用学习者的不受约束的语音话语来实现愿景。该项目将有助于科学知识构成L2可理解的语音，了解个性化客观的语音可理解性反馈如何影响L2语音发展，并创建语音/观念/信号处理措施的基础集合，以及ASR/DNN驱动器测量，以评估扬声器的智能和确定该技术的有效措施，以表现出nsf的高效方法。使用基金会的智力优点和更广泛的影响评估标准进行评估。

项目成果

FLUENCY BENCHMARKS AND IMPACTS OF PRACTICE WITH INSTANTANEOUS ASSESSMENT ON INTERNATIONAL TEACHING ASSISTANTS’ SPEECH RATE AND PAUSE UNITS

国际助教的流利度基准和即时评估实践的影响——语速和停顿单位

• DOI: 10.31274/psllt.15711
• 发表时间: 2023
• 期刊: Iowa State University Digital Press
• 作者: Hirschi, Kevin;Kang, Okim;Hansen, John;Looney, Stephen D
• 通讯作者: Looney, Stephen D

Characterization and normalization of second language speech intelligibility through lexical stress, speech rate, rhythm, and pauses

通过词汇重音、语速、节奏和停顿来表征和规范第二语言语音清晰度

• DOI: 10.1121/10.0016224
• 发表时间: 2022
• 期刊: The Journal of the Acoustical Society of America
• 作者: Kang, Okim;Hirschi, Kevin;Hansen, John H.;Looney, Stephen
• 通讯作者: Looney, Stephen