RI: Small: Collaborative Research: Developing Golden Speakers for Second-Language Pronunciation Training

RI：小型：合作研究：开发第二语言发音训练的黄金音箱

• 批准号: 1619212
• 负责人: Ricardo Gutierrez-Osuna
• 金额: $ 18万
• 依托单位: Texas A&M Engineering Experiment Station
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2016
• 资助国家: 美国
• 起止时间: 2016-09-01 至 2022-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1619212&HistoricalAwards=false
• 关键词: RI; Small; Collaborative; Research; Developing

People who learn a second language (L2) as adults often speak with a persistent foreign accent. This can make them less intelligible, more subject to discrimination, and less confident when interacting with others. Surprisingly, though, L2 learners rarely receive formal training in pronunciation, in part because effective training must be customized to meet each learner's individual needs. To address this gap, the investigators propose to develop algorithms to synthesize a personalized "golden speaker" for each learner: his or her own voice but with a native accent. The rationale is that, by listening to their own golden speaker, learners can more easily perceive differences between their actual and ideal pronunciations. This work focuses on developing the technology for golden speakers, which the investigators plan to evaluate in the future as a new tool for pronunciation learning systems. As such, this research can benefit a large number of workers in the US who are non-native speakers of English, particularly in higher education, health care and the technology sector. The project also provides opportunities for graduate and undergraduate students to conduct research in a multi-disciplinary team with expertise in signal processing, machine learning, and language acquisition. Two types of golden-speaker model are proposed. The first type is based on a reformulation of parametric statistical models for voice conversion, where instead of force-aligning source (native) and target (non-native) frames, they are matched based on their phonetic similarity. Several similarity metrics are proposed, from vocal-tract-length normalization to deep auto-encoders. The second type is based on a sparse representation of speech, which models individual frames as linear combinations of phonetic anchors. This requires new techniques to transform the constellation of anchors in the L2 speech to match the structure of native anchors (e.g., pairwise distances). Two types of evaluation are proposed for the golden-speaker models: their ability to interpolate phones not included in the learner's inventory, and the accent, intelligibility and comprehensibility of the resulting speech, as rated by native English listeners. For this purpose, the investigators propose to collect a large speech corpus from multiple Spanish and Korean learners of English and Indian speakers of English, each at different levels of English proficiency.

学习第二语言（L2）的人经常以持久的外国口音说话。 这可以使它们不那么理解，更受歧视的影响，并且在与他人互动时也不太自信。但是，令人惊讶的是，L2学习者很少接受发音的正规培训，部分原因是必须定制有效的培训以满足每个学习者的个人需求。 为了解决这一差距，调查人员建议开发算法，为每个学习者综合一个个性化的“金说话者”：他或她自己的声音，但带有本地口音。理由是，通过聆听自己的黄金演讲者，学习者可以更容易地感知其实际发音和理想发音之间的差异。这项工作着重于开发黄金扬声器的技术，研究人员计划将来评估该技术作为发音学习系统的新工具。因此，这项研究可以使美国的大量工人受益于非母语英语，尤其是高等教育，医疗保健和技术领域。该项目还为研究生和本科生提供了在具有信号处理，机器学习和语言获取方面专业知识的多学科团队中进行研究的机会。提出了两种类型的黄金扬声器模型。第一种类型是基于语音转换的参数统计模型的重新制定，其中它们不是使用力对准源（本机）和目标（非本地）帧，而是根据其语音相似性匹配的。 提出了几个相似性指标，从声音累积的归一化到深度自动编码器。第二种类型是基于语音的稀疏表示，该语音将单个帧模拟为语音锚的线性组合。 这需要新技术来改变L2语音中锚固的星座，以匹配天然锚的结构（例如，成对距离）。提出了针对黄金演讲者模型的两种类型的评估：它们的插入手机未包含在学习者库存中的能力，以及由本地英语听众评价的结果的口音，清晰度和可理解性。 为此，调查人员建议从多个西班牙语和韩国英语和印度英语说话者那里收集大量的语音语料库，每个人都具有不同的英语水平。