Control strategies for articulatory speech synthesis for natural user interfaces

自然用户界面的发音语音合成控制策略

• 批准号: 506576-2017
• 负责人: Fels, Sidney
• 金额: $ 14.34万
• 依托单位: University of British Columbia
• 依托单位国家: 加拿大
• 项目类别: Strategic Projects - Group
• 财政年份: 2018
• 资助国家: 加拿大
• 起止时间: 2018-01-01 至 2019-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=650584
• 关键词: Control; strategies; articulatory; speech; synthesis

We propose to advance the state of the art of articulatory speech synthesis research by building and evaluating neural activation-based models to drive a parameterized, 3D biomechanical-articulatory speech synthesizer. Our industry partner, CTF (http://www.ctfmeg.com) provides expertise in magnetoencephalography (MEG) and multimodal sensing of the brain and neural pathways, contributing to developing novel low degree-of-freedom (DoF) speech control strategies. The proposed research targets ICT - Human Interaction with Digital Information, as we focus on fundamental aspects of speech-based natural user interfaces. We plan to work backwards from our biomechanically driven speech synthesizer to determine control strategies that take advantage of the coupling afforded by biomechanics to reduce the DoF needed for producing natural sounding speech. Creating low DoF control representations is an important step for creating talking heads that move and sound natural for avatar-type interfaces needed for natural user interfaces. ** We have three central goals for this project: 1) improving our existing biomechanics-to-speech model, 2) developing neural control topologies, and 3) building control models for the coupled biomechanical-neural system. These goals will converge to synthesize a neurally controlled talking head for use in natural user interface evaluations. As a capstone demonstration, we will explore having a person speak in real-time using the neurally controlled talking head while in the MEG.** Our team of established researchers span the fields of expertise for this project. Drs. Fels, Stavness and Gick have published together on the biomechanics of the oral, pharyngeal and laryngeal complex and articulatory control of speech. CTF engineers have over a decade of experience advancing the state of the art of MEG for sensing neural activation. Our team's world-class modeling, simulation, HCI, speech synthesis control and MEG expertise provides a leading-edge opportunity for HQP to learn techniques for 3D physics-based simulation, advanced speech control, machine learning and neural sensing.

我们建议通过构建和评估基于神经激活的模型来驱动参数化的3D生物力学 - 定性语音合成器，以推动发音语音综合研究的艺术状态。我们的行业合作伙伴CTF（http://www.ctfmeg.com）提供了磁脑摄影（MEG）和大脑和神经途径的多模式感测的专业知识，从而有助于发展新型的低自由度（DOF）语音控制策略。拟议的研究针对ICT - 与数字信息的人类互动，因为我们专注于基于语音的自然用户界面的基本方面。我们计划从生物力学驱动的语音合成器中向后进行倒退，以确定利用生物力学提供的耦合以减少产生自然发声语音所需的DOF的控制策略。创建低DOF控制表示形式是创建会说话头的重要步骤，这些说话头是自然用户界面所需的阿凡达型接口的移动和自然声音的重要步骤。 **我们有三个中心目标：1）改善现有的生物力学对语音模型，2）开发神经控制拓扑以及3）耦合生物力学神经系统的建筑控制模型。这些目标将融合到合成一个神经控制的通话头，以用于自然用户界面评估。作为一个顶峰演示，我们将在MEG中使用神经控制的说话头实时实时讲话。**我们的知名研究人员团队涵盖了该项目的专业知识领域。博士。 Fels，Stavness和Gick共同发表了关于口腔，咽和喉部复合物的生物力学以及语音的发音控制。 CTF工程师拥有十多年的经验，可以推动MEG的最新动物感知神经激活。我们团队的世界一流的建模，模拟，HCI，语音综合控制和MEG专业知识为HQP提供了用于学习基于3D物理学的模拟，高级语音控制，机器学习和神经传感的技术的领先机会。