EMG Voice Restoration

肌电图语音恢复

• 批准号: 10009728
• 负责人: Gianluca De Luca
• 金额: $ 50.63万
• 依托单位: ALTEC, INC.
• 依托单位国家: 美国
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-05-16 至 2023-03-31
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/10009728
• 关键词: Acoustics; Affinity; Algorithm Design; Algorithms; Architecture; Articulators; Augmentative and Alternative Communication; Cancer Survivor; Clinical; Communication; Computer software; Computers; Cosmetics; Custom; Data; Devices; Discrimination; Electrolarynx; Engineering; Eye Movements; Face; Future; General Hospitals; Genetic Transcription; Gestures; Goals; Hand; Human; Impairment; Individual; Intuition; Laryngectomee; Laryngectomy; Life; Malignant neoplasm of larynx; Massachusetts; Measures; Mental Health; Methods; Modeling; Motor; Muscle; Neck; Operative Surgical Procedures; Oral cavity; Orthography; Participant; Patient Self-Report; Performance; Persons; Phase; Production; Protocols documentation; Provider; Quality of life; Series; Signal Transduction; Small Business Innovation Research Grant; Social Interaction; Socialization; Speech; Speech Disorders; Stress; Surface; System; Tablet Computer; Technology; Testing; Text; Time; Training; Transcript; Translations; Variant; Vision; Voice; Work; base; design; digital; handheld mobile device; improved; indexing; lexical; malignant oropharynx neoplasm; neural network; phrases; prototype; restoration; sensor; showing emotion; social; speech recognition; success; time use; vocalization; wearable sensor technology

Nearly 7.5 million people live without the ability to vocalize effectively. Existing augmentative and alternative communication (AAC) technology provides some function for these individuals, typically by converting physical gestures, eye movements or text into words that can be acoustically synthesized or visually displayed. However, a key limitation of these devices is that they do not involve natural mechanisms of speech production and therefore can be less intuitive as substitutes for the human vocal system. Consequently, they can suffer from lexical ambiguity, lack of emotional expression, and difficulty in conveying intent. There remains an unmet need to restore the natural mechanisms of speech production for the vocally impaired. To meet this need, we propose to develop a first-of-its-kind AAC system that restores personalized, prosodic, near real-time vocalization based on surface electromyographic (sEMG) signals produced during subvocal (i.e., silently mouthed) speech. In Phase I, we demonstrated the ability to recognize orthographic content and categorize emphatic stress between phrases subvocalized by (n=4) control and (n=4) post-laryngectomy participants with a 96.3% word recognition rate and 91.2% emphatic stress discrimination rate, respectively. Subvocal speech corpus transcripts were synthesized into prosodic speech using personalized, digital voices unique to each participant, then evaluated by naïve listeners (n=12). Listeners consistently rated our sEMG-based digital voice as having greater intelligibility, acceptability, emphasis discriminability and vocal affinity than the state-of-the-art electrolarynx (EL) speech aid used by laryngectomees. Having achieved these capabilities with lengthy post-processing of single phrases, we now aim to advance this technology in Phase II by solving the more fundamental challenges of transcribing prosodic speech and tracking variations in intonation and timing in near-real-time to restore conversational interactions in everyday life. To achieve this goal, our team of engineers at Altec Inc. is partnering with the world’s leading provider of personalized digitized voice for AAC (VocaliD, Inc), and world-class laryngeal cancer clinical experts (Massachusetts General Hospital) to develop algorithms for transcribing prosodic speech and tracking variations in intonation and timing throughout narratives, monologues and conversations (Aim 1); design MyoVoice™ system for near real-time mobile use (Aim 2); and evaluate the prototype system for conversational efficacy (Aim 3). Our milestone is to demonstrate within-subject improvements in ease-of-use, functional efficacy, and social reception amongst post-laryngectomy participants using our sEMG-based digital voice when compared to their typical EL speech aid. The final deliverable will consist of a single 4-contact sensor veneer and cross-platform, near-real-time mobile software that can operate on an AAC tablet or mobile device. Once commercialized, our vision for the future of this device is for a person—who is facing the devastating need to undergo laryngectomy—to have their voice banked and subvocal models trained such that immediately following surgery, they can receive a custom MyoVoice™ system to restore their original voice.

近 750 万人无法有效发声。 通信（AAC）技术为这些人提供了一些功能，通常通过转换 将身体姿势、眼球运动或文本转化为可以通过声音合成或视觉显示的文字。 然而，这些设备的一个关键限制是它们不涉及语音产生的自然机制 因此，作为经过测试的人类发声系统的替代品，它们可能不太直观，因此可能会受到影响。 词汇歧义、缺乏情感表达以及难以传达意图仍然存在未满足的需求。 恢复声乐障碍者言语产生的自然机制 为了满足这一需求，我们建议。 开发首个 AAC 系统，可恢复基于个性化、韵律、近实时的发声 在默声（即无声说话）同相讲话期间产生的表面肌电 (sEMG) 信号。 I，我们展示了识别拼写内容并对短语之间的重音进行分类的能力 由 (n=4) 对照组和 (n=4) 喉切除术后参与者默读，单词识别率为 96.3% 合成的潜声语音语料库转录本的强调重音辨别率分别为 91.2%。 使用每个参与者独特的个性化数字声音转化为韵律语音，然后由 naïve 进行评估 听众 (n=12) 一致认为我们基于 sEMG 的数字语音具有更高的清晰度， 与最先进的电喉 (EL) 语音辅助设备相比，其可接受性、重点辨别力和声音亲和力 通过对单个短语进行长时间的后处理，我们实现了这些功能。 现在的目标是通过解决转录的更根本挑战来推进第二阶段的技术 韵律语音并近实时跟踪语调和时间的变化以恢复会话 为了实现这一目标，我们 Altec Inc. 的工程师团队正在与 全球领先的 AAC 个性化数字化语音提供商 (VocaliD, Inc) 和世界一流的喉音 癌症临床专家（马萨诸塞州总医院）开发用于转录韵律语音的算法 跟踪叙述、独白和对话中语调和时间的变化（目标 1）； 设计用于近实时移动使用的 MyoVoice™ 系统（目标 2）并评估原型系统； 对话功效（目标 3）是展示受试者内部在易用性方面的改进， 使用我们基于表面肌电图的数字技术评估喉切除术后参与者的功能功效和社会接受度 与典型的 EL 语音辅助设备相比，最终交付的产品将包含一个 4 接触式传感器。 veneer 和跨平台、近乎实时的移动软件，可以在 AAC 平板电脑或移动设备上运行。 一旦商业化，我们对该设备未来的愿景是针对面临毁灭性需求的人 接受喉切除术——对他们的声音进行存储并训练默音模型，以便立即 手术后，他们可以获得定制的 MyoVoice™ 系统来恢复原来的声音。