Next-Generation Expressive Personalized Voices for Speech-Generating Devices

用于语音生成设备的下一代富有表现力的个性化声音

• 批准号: 10547241
• 负责人: H TIMOTHY Bunnell
• 金额: $ 27.58万
• 依托单位: SYNFONICA, LLC
• 依托单位国家: 美国
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-08-15 至 2024-08-14
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/10547241
• 关键词: ALS patients; Adoption; Adult; Age; Algorithms; Amyotrophic Lateral Sclerosis; Augmentative and Alternative Communication; Characteristics; Child; Child Health; Client; Depressed mood; Disease; Dysarthria; Emotions; Encapsulated; Evaluation; Female; Generations; Goals; Government; Human; Hybrids; Individual; Knowledge; Laboratory Research; Learning; Linguistics; Machine Learning; Methods; Modeling; Network-based; Neurodegenerative Disorders; Onset of illness; Outcome; Output; Persons; Phase; Process; Production; Reading; Records; Rehabilitation therapy; Risk; Running; Services; Speech; Structure; Surveys; System; Technology; Text; Training; Voice; Voice Quality; base; commercial application; communication device; deep neural network; design; experience; experimental study; improved; knowledge base; machine learning algorithm; male; mimetics; next generation; novel; sound; success; virtual vocal tract

Project Summary/Abstract The creation of personalized synthetic voices has wide application in medical/rehabilitation settings for pa- tients who rely on a speech-generating device (SGD) for communication. One common application is voice banking, wherein a person who risks losing their voice, such as somebody with a neurodegenerative disease like Amyotrophic Lateral Sclerosis (ALS), records their own speech before the onset of disease-related dysar- thria for later use in an SGD that mimics their natural speech characteristics. While the technology underlying the creation of such personalized synthetic voices is growing in maturity and adoption by SGD users, it still suf- fers from two primary limitations: a lack of expressiveness and a burdensome amount of recording needed to create highly natural-sounding voices. The proposed project aims to remedy this situation by marrying the ma- chine-learning technology behind ModelTalker, a pioneering voice-banking text-to-speech service developed at Nemours Children’s Health, with the knowledge-based technology underlying Synfony, a rule-based text-to- speech system developed by Synfonica LLC, which is capable of generating a variety of speech styles and ex- pressive modes. The expert knowledge built into Synfonica will be used to design an optimal set of sentences for voice bankers to record, and its algorithms for the generation of natural-sounding prosody in different modes and styles will be integrated into ModelTalker’s machine-learning algorithms, creating a hybrid system that embraces the best qualities of both approaches. The new text-to-speech (TTS) system resulting from this project will (a) require a minimal amount of recorded speech from the voice banker, (b) accurately capture their vocal identity, and (c) be structured such that new expressive modes and speech styles can be added easily without additional recording. The feasibility of the project will be demonstrated by recording the voices of an adult male, an adult female, and a child, and generating TTS voices that can speak in three expressive modes (neutral, happy, and sad). Perceptual experiments will be run to evaluate their intelligibility, naturalness, suc- cess in capturing the vocal identity of the speaker, and the appropriateness of their expressive modes. In gen- eral, the project will be a major step forward in enabling the users of personalized synthetic voices to express their emotions and intentions.

项目概要/摘要 个性化合成声音的创建在医疗/康复环境中具有广泛的应用。 依靠语音生成设备 (SGD) 进行通信的一种常见应用是语音。 银行业，指有失声风险的人，例如患有神经退行性疾病的人 像肌萎缩侧索硬化症（ALS）一样，在与疾病相关的肌无力发作之前记录自己的言语。 thria 供以后在模仿其自然语音特征的 SGD 中使用，而该技术是底层的。 这种个性化合成声音的创建正在日益成熟并被 SGD 用户采用，但它仍然满足 存在两个主要限制：缺乏表现力和需要大量的记录 拟议的项目旨在通过将ma-结合起来来纠正这种情况。 ModelTalker 背后的中文学习技术是一项开创性的语音银行文本转语音服务，由 Nemours Children's Health，采用基于知识的 Synfony 技术，这是一种基于规则的文本到 Synfonica LLC 开发的语音系统，能够生成多种语音风格和前 Synfonica 内置的专业知识将用于设计一组最佳句子。 供语音银行人员录制，及其生成不同语言的自然发音韵律的算法 模式和风格将集成到 ModelTalker 的机器学习算法中，创建一个混合系统 由此产生的新文本转语音 (TTS) 系统。 项目将 (a) 需要语音银行人员录制最少量的语音，(b) 准确捕捉 他们的声音特征，以及（c）结构化以便可以轻松添加新的表达模式和演讲风格 无需额外录音，该项目的可行性将通过录制一个人的声音来证明。 成年男性、成年女性和儿童，并生成可以三种表达模式说话的 TTS 语音 （中性、快乐和悲伤）将进行感知实验来评估它们的可理解性、自然性、成功度。 捕捉说话者的声音特征及其表达方式的适当性的过程。 总之，该项目将是在使个性化合成声音的用户能够表达声音方面向前迈出的重要一步 他们的情绪和意图。