RI: Small: Creating Text-to-Speech Synthesis for Low Resource Languages

RI：小型：为低资源语言创建文本到语音合成

• 批准号: 1717680
• 负责人: Julia Hirschberg
• 金额: $ 50万
• 依托单位: Columbia University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-09-01 至 2021-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1717680&HistoricalAwards=false
• 关键词: RI; Small; Creating; Text; Speech

Recent advances in speech technology have resulted in wide use of Spoken Dialogue Systems (SDS) such as Siri (iPhone) and Google Assistant (Android). These systems support major improvements in information access by voice for High Resource Languages (HRLs) such as English, French, Mandarin, Japanese, and Spanish. For these languages, researchers have built dictionaries, parsers, part-of-speech taggers, language models, search engines, and machine translation engines to support speech technologies. However, there are ~6500 world languages, including Tagalog, Tamil, Swahili, Vietnamese and Pashto, many of which are spoken by millions of people, but which do not enjoy the computational resources necessary to build SDS. These are termed Low Resource Languages (LRLs). Speakers of LRLs do not benefit from the same communication and search capabilities speakers of HRLs do. In particular, there is little research and few resources supporting the development of Text-to-Speech Synthesis (TTS) systems to produce Siri-like speech for SDS in these languages. Furthermore, both commercial and research TTS systems also require large amounts of carefully recorded, single-speaker speech data, creating another major (and expensive) barrier to TTS development for LRLs. This work will create TTS systems in LRLs and, in the process, create and make available tools for others to create their own systems using "found" data - data recorded for other purposes or available on the web.New paradigms for TTS synthesis (parametric synthesis and the use of Deep Neural Nets) are now being developed which make it theoretically possible to build systems quickly and cheaply without recording large, special-purpose speech corpora, instead using data recorded for other purposes such as training speech recognizers. This work will investigate the use these techniques to produce TTS systems for LRL. Two major problems will be explored: 1) What are the best techniques to filter found data (removing data that is too loud, too noisy or disfluent, for example) to obtain intelligible and natural-sounding results? 2) Can basic prosodic features of LRLs such as phrasing and emphasis be identified, using crowdsourcing and tools developed for HRLs? Pilot studies on English have revealed that more natural and intelligible voices can be created by using subsets of the data selected on features such as pitch variation and level of articulation. These methods will be tested on LRLs such as Turkish, Amharic, and Telugu. Evaluations will be made in terms of intelligibility and naturalness both automatically and using crowdsourcing techniques with native speakers of each language. The ultimate goal of this exploratory work will be to test these techniques on a broad variety of LRLs which have been collected for purposes of developing speech recognizers.

语音技术的最新进展导致了语音对话系统 (SDS) 的广泛使用，例如 Siri (iPhone) 和 Google Assistant (Android)。 这些系统支持通过语音访问英语、法语、普通话、日语和西班牙语等高资源语言 (HRL) 的信息访问的重大改进。 对于这些语言，研究人员构建了词典、解析器、词性标注器、语言模型、搜索引擎和机器翻译引擎来支持语音技术。 然而，世界上有约 6500 种语言，包括他加禄语、泰米尔语、斯瓦希里语、越南语和普什图语，其中许多语言有数百万人使用，但它们不具备构建 SDS 所需的计算资源。这些被称为低资源语言（LRL）。 LRL 的发言者无法从 HRL 的发言者所享有的相同通信和搜索功能中受益。 特别是，支持开发文本转语音合成 (TTS) 系统以在这些语言中为 SDS 生成类似 Siri 的语音的研究和资源很少。 此外，商业和研究 TTS 系统还需要大量仔细记录的单扬声器语音数据，这为 LRL 的 TTS 开发造成了另一个主要（且昂贵）障碍。 这项工作将在 LRL 中创建 TTS 系统，并在此过程中为其他人创建并提供可用的工具，以便使用“找到的”数据（为其他目的记录的数据或在网络上提供的数据）创建自己的系统。 TTS 合成的新范例（参数化）目前正在开发中，这使得在理论上可以快速且廉价地构建系统，而无需记录大型的专用语音语料库，而是使用为其他目的（例如训练语音识别器）记录的数据。 这项工作将研究如何使用这些技术来为 LRL 制作 TTS 系统。 将探讨两个主要问题：1）过滤找到的数据（例如，删除太大声、太嘈杂或不流畅的数据）以获得可理解且听起来自然的结果的最佳技术是什么？ 2) 能否使用众包和为 HRL 开发的工具来识别 LRL 的基本韵律特征，例如措辞和强调？ 对英语的试点研究表明，通过使用根据音调变化和发音水平等特征选择的数据子集，可以创建更自然、更容易理解的声音。 这些方法将在土耳其语、阿姆哈拉语和泰卢固语等 LRL 上进行测试。 将自动并使用众包技术与每种语言的母语人士一起对清晰度和自然度进行评估。 这项探索性工作的最终目标是在各种 LRL 上测试这些技术，这些 LRL 是为了开发语音识别器而收集的。

项目成果

Characteristics of Text-to-Speech and Other Corpora

文本转语音和其他语料库的特征

• DOI: 10.21437/speechprosody.2018-140
• 发表时间: 2018-06-13
• 期刊: Speech Prosody 2018
• 作者: Erica Cooper;E. Li;Julia Hirschberg
• 通讯作者: Julia Hirschberg

Adaptation and Frontend Features to Improve Naturalness in Found-Data Synthesis

适应和前端功能可提高发现数据合成的自然度

• DOI: 10.21437/speechprosody.2018-160
• 发表时间: 2018-06
• 期刊: Speech Prosody 2018
• 作者: Cooper, Erica;Hirschberg, Julia
• 通讯作者: Hirschberg, Julia

Subset Selection, Adaptation, Gemination and Prosody Prediction for Amharic Text-to-Speech Synthesis

阿姆哈拉语文本转语音合成的子集选择、适应、双生和韵律预测

• DOI: 10.21437/ssw.2019-37
• 发表时间: 2019-09
• 期刊: 10th ISCA Speech Synthesis Workshop
• 作者: Tesfaye Biru, Elshadai;Tofik Mohammed, Yishak;Tofu, David;Cooper, Erica;Hirschberg, Julia
• 通讯作者: Hirschberg, Julia

Utterance Selection for Optimizing Intelligibility of TTS Voices Trained on ASR Data

用于优化基于 ASR 数据训练的 TTS 语音清晰度的话语选择

• DOI: doi: 10.21437/interspeech.2017-465
• 发表时间: 2017-01
• 期刊: Interspeech 2017
• 作者: Erica Cooper; Xinyue Wang
• 通讯作者: Xinyue Wang

Adaptation and Frontend Features to Improve Naturalness in Found-Data Synthesis

适应和前端功能可提高发现数据合成的自然度

• 发表时间: 2018-06
• 期刊: Proceedings of Speech Prosody 2018
• 作者: Cooper, Erica;Hirschberg, Julia
• 通讯作者: Hirschberg, Julia