RI: Medium: Deep Neural Networks for Robust Speech Recognition through Integrated Acoustic Modeling and Separation

RI：中：通过集成声学建模和分离实现鲁棒语音识别的深度神经网络

• 批准号: 1409431
• 负责人: Eric Fosler-Lussier
• 金额: $ 79.81万
• 依托单位: Ohio State University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2014
• 资助国家: 美国
• 起止时间: 2014-06-01 至 2019-05-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1409431&HistoricalAwards=false
• 关键词: RI; Medium; Deep; Neural; Networks

Over the last decade, speech recognition technology has become steadily more present in everyday life, as seen by the proliferation of applications including mobile personal agents and transcription of voicemail messages. Performance of these systems, however, degrades significantly in the presence of background noise; for example, using speech recognition technology in a noisy restaurant or on a windy street can be difficult because speech recognizers confuse the background noise with linguistic content. Compensation for noise typically involves preprocessing the acoustic signal to emphasize the speech signal (i.e. speech separation), and then feeding this processed input into the recognizer. The innovative approach in this project is to train the recognition and separation systems in an integrated manner so that the linguistic content of the signal can inform the separation, and vice versa. Given the impact of the recent resurgence of Deep Neural Networks (DNNs) in speech processing, this project seeks to make DNNs more resistant to noise by integrating speech separation and speech recognition, exploring three related areas. The first research area seeks to stabilize input to DNNs by combining DNN-based suppression and acoustic modeling, integrating masking estimates across time and frequency, and using this information to improve reconstruction of speech from noisy input. The second area seeks to examine a richer DNN structure, using multi-task learning techniques to guide the construction of DNNs better at performing all tasks and where layers have meaningful structure. The final research area examines ways to adapt the spurious output of DNN acoustic models given acoustic noise. With the focus of integrating speech separation and recognition, the project will be evaluated both by measuring speech recognition performance, as well as metrics that are more closely related to human speech perception. This will ensure a broader impact of this research by providing insights not only to speech technology but also facilitating the design of next-generation hearing technology in the long run.

在过去的十年中，语音识别技术在日常生活中的应用越来越广泛，从移动个人代理和语音邮件消息转录等应用程序的激增就可以看出这一点。然而，在存在背景噪声的情况下，这些系统的性能会显着下降。例如，在嘈杂的餐厅或有风的街道上使用语音识别技术可能会很困难，因为语音识别器会将背景噪音与语言内容混淆。噪声补偿通常涉及预处理声学信号以强调语音信号（即语音分离），然后将经过处理的输入馈送到识别器中。 该项目的创新方法是以集成的方式训练识别和分离系统，以便信号的语言内容可以通知分离，反之亦然。鉴于最近深度神经网络 (DNN) 在语音处理领域的复兴所带来的影响，该项目旨在通过集成语音分离和语音识别来增强 DNN 的抗噪声能力，并探索三个相关领域。 第一个研究领域旨在通过结合基于 DNN 的抑制和声学建模，集成跨时间和频率的掩蔽估计，并使用此信息来改进从噪声输入中重建语音，从而稳定 DNN 的输入。 第二个领域旨在研究更丰富的 DNN 结构，使用多任务学习技术来指导 DNN 的构建更好地执行所有任务以及层具有有意义的结构。 最后一个研究领域研究在给定声学噪声的情况下调整 DNN 声学模型的杂散输出的方法。 该项目的重点是集成语音分离和识别，将通过测量语音识别性能以及与人类语音感知更密切相关的指标来进行评估。 这将确保这项研究产生更广泛的影响，不仅提供对语音技术的见解，而且从长远来看也促进下一代听力技术的设计。