On the Development of Real Time Lip Reading System for Word Recognition in Noisy Circumstances

噪声环境下实时唇读文字识别系统的研制

• 批准号: 12650427
• 负责人: SUGAHARA Kazunori
• 金额: $ 2.18万
• 依托单位: Tottori University
• 依托单位国家: 日本
• 项目类别: Grant-in-Aid for Scientific Research (C)
• 财政年份: 2000
• 资助国家: 日本
• 起止时间: 2000 至 2001
• 项目状态: 已结题
• 来源: https://kaken.nii.ac.jp/grant/KAKENHI-PROJECT-12650427/
• 关键词: Lip Reading; Word Recognition; Noisy Circumstance; FPGA; 音声認識; 唇形状の時間変化; 動的輪郭モデル; 隠れマルニフモデル; 隠れマルコフモデル

Robotic technology had been developed to provide substitution of simple human operations. However, the importance of assistant behavior according to the human commands instead of the stand-alone behavior has been recognized in recent years. In such cases, a variety of communication methods between human and robots can be considered. The word recognition technique had been developed in several researchers and in calm situation it can be used for practical use. But in noisy circumstances, they are not still applicable and have a number of problems to solve. For word recognition method in noisy circumstances, the lip reading method is investigated in our research. The lip reading method does not require the additional training to utilize it.In this research, the video images of human speaking are input to the system and the lip shape movements are automatically extracted to provide word recognition parameters. For these automatic extractions of lip shapes of the NTSC video signal rate are realized by hardware realization of the active contour model techniques. For future works, the development of the lip area separation technique from complex background is remained.

机器人技术的发展是为了替代简单的人类操作。然而，近年来，人们已经认识到根据人类命令进行辅助行为而不是独立行为的重要性。在这种情况下，可以考虑人与机器人之间的多种通信方法。单词识别技术已经由几位研究人员开发出来，在平静的情况下可以用于实际用途。但在嘈杂的环境中，它们仍然不适用，并且有许多问题需要解决。对于噪声环境下的单词识别方法，我们研究了唇读方法。唇读方法不需要额外的训练来使用它。在这项研究中，将人类说话的视频图像输入到系统中，自动提取唇形动作以提供单词识别参数。对于NTSC视频信号速率的嘴唇形状的这些自动提取是通过活动轮廓模型技术的硬件实现来实现的。对于未来的工作，仍保留复杂背景中唇部区域分离技术的发展。

项目成果

鷲尾昌宣: "動画像処理を目指した動的輪郭モデルのハードウェア化について"電子情報通信学会技術研究報告. DSP2001-83. 23-30 (2001)

Masanori Washio：“运动图像处理的动态轮廓模型的硬件实现”IEICE技术研究报告23-30（2001）。

菅原一孔: "パーソナルコンピュータ上での読唇システムの実時間実現"計測自動制御学会論文誌. 36・12. 1145-1151 (2000)

Kazunori Sukawara：“在个人计算机上实时实现唇读系统”，仪器与控制工程师学会汇刊 36・12（2000 年）。

Naoshi KAKITA, Kazunori SUGAHARA, Ryosuke KONISHI: "On the Lip Area Extraction Method based on the Color Information"Technical report of the institute of electronics, information and communication engineers. Vol. DSP2001-83. 17-22 (2001)

Naoshi KAKITA、Kazunori SUGAHARA、Ryosuke KONISHI：“基于颜色信息的唇部区域提取方法”电子信息通信工程师研究所的技术报告。

Kazunori SUGAHARA: "Personal Computer Based Real Time Lip Reading System"2000 5th.Int.Conf. Signal Processing. 1341-1346 (2000)

Kazunori SUGAHARA：“基于个人计算机的实时唇读系统”2000 年第 5 届国际会议。

菅原一孔: "パーソナルコンピュータ上での読唇システムの実時間実現"計測自動制御学会論文誌. 36. 1145-1151 (2000)

Kazunori Sukawara：“在个人计算机上实时实现唇读系统”《仪器与控制工程师学会汇刊》36. 1145-1151 (2000)。