Wearable silent speech technology to enhance impaired oral communication

可穿戴式无声语音技术可增强受损的口语交流

• 批准号: 10456297
• 负责人: Jun Wang
• 金额: $ 56.34万
• 依托单位: UNIVERSITY OF TEXAS AT AUSTIN
• 依托单位国家: 美国
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-08-15 至 2024-07-31
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/10456297
• 关键词: Acoustics; Age; Alaryngeal Speech; Algorithms; Articular Range of Motion; Articulation; Articulators; Characteristics; Communication; Data; Data Collection; Development; Devices; Electrolarynx; Electromagnetics; Enhancement Technology; Esophageal Speech; Excision; Gender; Goals; Gold; Health; Hoarseness; Human; Impairment; Individual; Knowledge; Laryngeal Prosthesis; Laryngectomee; Laryngectomy; Larynx; Life; Lip structure; Machine Learning; Magnetism; Malignant neoplasm of larynx; Maps; Measures; Mechanics; Mental Depression; Motion; Motor; Movement; Output; Participant; Patients; Pattern; Performance; Persons; Population; Positioning Attribute; Production; Quality of life; Research; Robotics; Science; Self-Help Devices; Speech; Speech Disorders; Speech Sound; Speech Synthesizers; Speech Therapy; Speed; Technology; Testing; Time; Tongue; Tracer; Tracheoesophageal Speech; United States; Voice; Voice Disorders; Voice Quality; Wireless Technology; alternative communication; auditory feedback; base; design; experimental study; improved; innovation; kinematics; language training; machine learning algorithm; millisecond; new technology; novel; oral communication; preservation; prototype; social; social exclusion; sound; speech synthesis; tool; vibration; visual feedback; vocal cord; wearable device; wireless; wireless electronic

Project Summary/Abstract The long-term objectives of this project are to obtain a deeper understanding how articulatory movement patterns are mapped to speech particularly when there is no vocal fold vibration (silent speech) and then to develop a novel, wearable assistive technology called silent speech interface (SSI) to assist the impaired oral communication for individuals in need (e.g., individuals after laryngectomy, surgical removal of larynx to treat advanced laryngeal cancer). Designed for daily use, the SSI contains a wearable magnetic device and a small camera for tongue and lip motion tracking, respectively, and an articulation-to-speech synthesizer to output natural sounding speech that preserves the speaker’s voice characteristics. Specific Aims of the proposal include to (1) determine the articulatory patterns of normal (vocalized) and silent speech, produced by both healthy talkers and people after laryngectomy, (2) develop a wearable, wireless magnetic device for real-time tongue and lip motion tracking, and (3) synthesize speech from articulation directly. There are currently limited alternative communication options for people who have undergone laryngectomy. These options include esophageal speech, tracheo-esophageal speech, and use of an artificial larynx (or electrolarynx). These solutions are either invasive or difficult to use, and all of them result in a hoarse or mechanical/robotic sounding voice, which can be difficult to understand. In contrast, the SSI in this application is non-invasive, easy-to-use, and produces natural sounding speech and may even preserve the patient’s voice identity. We have exciting preliminary results that support the feasibility of the project including that (1) we have recently developed a wireless magnetic device for tongue motion, and (2) we have demonstrated real- time articulation-to-speech synthesis with a 90% word accuracy (judged by a human listener). In this project, we will further reduce the size of the wireless device and make it wearable and conduct articulation-to-speech algorithms by studying 30 participants after laryngectomy and 30 age- and gender-matched healthy controls. If successful, the proposed research will enhance human health by making an impact on individuals after laryngectomy and potentially to a broader range of other speech and voice disorders. In addition, the technology will have an impact to the speech science field by providing a fist-time-ever tool for potential large- scale tongue motion data collection and have a variety of broader implications including visual feedback-based secondary language training and speech therapy, which may benefit millions of people with motor speech deficits in the United States.

项目摘要/摘要 该项目的长期目标是获得更深入的了解如何旋律的运动 图案被映射到语音，特别是在那时，然后进行表达折叠振动（无声语音），然后 开发一种名为“静音语音界面（SSI）”的小说，可穿戴的辅助技术，以帮助口服受损 有需要的个体的交流（例如，喉切除术后的个体，手术去除喉部以信任治疗 晚期喉癌）。 用于舌头和唇部运动跟踪的摄像头，分别用于输出的艺术对语音合成器 自然发声的演讲可以保留说话者的声音特征。 包括（1）确定正常（发声）和无声言语的关节性模式，两者都产生 喉切除术后健康的说话者和人，（2）开发一种可穿戴的无线磁性设备，以实时 舌头和嘴唇运动跟踪，以及（3）直接从人工制作中综合语音。 对于接受喉切除术的人来说，这是有限的替代沟通选择。 这些选项包括食管语音，气管食道语音以及人工喉（OR）的使用 这些溶液是侵入性的，要么是使用 机械/机器人的声音，可以理解该应用程序中的SSI。 是无创的，易于使用的，并且会产生自然的发声，甚至可以保留患者的 语音身份。我们有令人兴奋的预制结果 我们最近开发了一种用于舌运动的无线磁性设备，（2）我们已经证明 时间到言论的时间综合，具有90％的单词精度（由人类听众判断）。 我们将进一步降低无线设备，并使其可穿戴和策略到语音 通过研究喉切除术和30个年龄和性别匹配的健康对照的算法 成功，拟议的研究将通过对个人产生影响之后的个人来增强人类健康 喉切除术，并有可能采用更广泛的语音和语音疾病 技术将通过为潜在的大型活动提供拳头，对语音科学领域产生影响 扩展舌头运动数据收集，并具有多种宽广的含义，包括基于视觉反馈 二级语言培训和语音疗法，这可能使数百万有运动言语的人受益 美国的缺陷。