Japanese Machine Lip-reading System Based on Human Lip-reading Method

日本基于人类唇读方法的机器唇读系统

• 批准号: 23700672
• 负责人: MIYAZAKI Tsuyoshi
• 金额: $ 1.91万
• 依托单位: Kanagawa Institute of Technology
• 依托单位国家: 日本
• 项目类别: Grant-in-Aid for Young Scientists (B)
• 财政年份: 2011
• 资助国家: 日本
• 起止时间: 2011 至 2013
• 项目状态: 已结题
• 来源: https://kaken.nii.ac.jp/grant/KAKENHI-PROJECT-23700672/
• 关键词: 機械読唇; 障害者支援; 画像処理; 画像認識; パターン認識; 聴覚障害者支援; 発話映像; 口形画像; 口形検出; 動画像認識; 聴覚障がい者支援

I have improved the detection rate of mouth shapes from Japanese utterance images, and proposed a recognition method of Japanese words. I aimed for more than 90% at the detection rate of mouth shapes, but it was finally with 82.7% by using the optical flow and the template matching. This shows that the detection rate was improved by 8.6% from the grant start. By the recognition of the Japanese utterance word, I proposed the calculation method as reliability in consideration of the length of utterance word, and 36 prefectures were recognized from 47 prefectures utterance images by the experiment. The recognition rate became approximately 76.6%. In future, I consider a method to improve accuracy to extract the period that the mouth shapes are formed.

我提高了日语语音图像中嘴形的检测率，并提出了日语单词的识别方法。我的目标是口型检测率超过90%，但通过使用光流和模板匹配，最终达到了82.7%。这表明检测率比资助开始时提高了 8.6%。通过对日语话语词的识别，提出了考虑话语词长度的可靠性计算方法，并通过实验从47个都道府县的话语图像中识别出36个都道府县。识别率变为约76.6%。将来，我会考虑一种提高提取嘴形形成时期的准确性的方法。

项目成果

口形ベースの機械読唇における単語認識手法の提案と評価

基于口型的机器唇读词语识别方法的提出与评价

• 发表时间: 2014
• 作者: 宮崎剛; 中島豊四郎
• 通讯作者: 中島豊四郎

Producing text and speech from video images of lips movement photographed in speaking Japanese by using mouth shape sequence code -An experimental system to communicate with hearing impaired persons -

使用口形序列代码，根据日语中拍摄的嘴唇运动视频图像生成文本和语音 -与听力障碍者交流的实验系统 -

• 发表时间: 2012
• 作者: Shiori Kawahata; Eiko Koyama; Tsuyoshi Miyazaki;Fujio Yamamoto
• 通讯作者: Fujio Yamamoto

A Detection Method of Basic Mouth Shapes from Japanese Utterance Images by Template Matching and Optical Flow

基于模板匹配和光流的日语语音图像基本嘴型检测方法

• 发表时间: 2012
• 作者: Tsuyoshi Miyazaki; Toyoshiro Nakashima;Naohiro Ishii
• 通讯作者: Naohiro Ishii

Analysis of Mouth Shape Deformation Rate for Generation of Japanese Utterance Images Automatically

自动生成日语语音图像的口形变形率分析

• 发表时间: 2014
• 作者: Tsuyoshi Miyazaki;Toyoshiro Nakashima
• 通讯作者: Toyoshiro Nakashima

Producing text and speech from video images of lips movement photographed in speaking Japanese by using mouth shape sequence code - An experimental system to communicate with hearing impaired persons -

使用口形序列代码，根据日语中拍摄的嘴唇运动视频图像生成文本和语音 - 与听力障碍者交流的实验系统 -

• 发表时间: 2012
• 作者: Shiori Kawahata; Eiko Koyama; Tsuyoshi Miyazaki;Fujio Yamamoto
• 通讯作者: Fujio Yamamoto