Study on Speech Recognition by Uniting Image Sequence and Acoustic Signal

图像序列与声音信号相结合的语音识别研究

• 批准号: 11680397
• 负责人: TAGAWA Norio
• 金额: $ 2.43万
• 依托单位: Tokyo Metropolitan University
• 依托单位国家: 日本
• 项目类别: Grant-in-Aid for Scientific Research (C)
• 财政年份: 1999
• 资助国家: 日本
• 起止时间: 1999 至 2000
• 项目状态: 已结题
• 来源: https://kaken.nii.ac.jp/grant/KAKENHI-PROJECT-11680397/
• 关键词: Speech recognition; 3-D lips shape; vowel recognition; word recognition; HMM; moire analysis; reflectance; region segmentation; 領域分離; 発語認識; 口形状; 3次元形状; モアレトポグラフィ

(1) Development of method for recovering 3-D lips shape from image sequence(2) We developed a method for extracting the absolute depth information from the depth information having an indefinite scale, which is obtained by a moire imaging, with a simple reflection model for an object surface. It was confirmed through simulations and real image experiments that by considering not only a diffuse reflectance but also a specular one, an unbiased depth can be computed.(3) Development of vision-based speech recognition method using 3-D lips shapeWe proposed a method of a vowel recognition and a word recognition using 3-D lips shape information. The advantage of the proposed method is that an invariant recognition against a relative angle between a face and a camera can be done and a recognition rate purely improves by applying the proposed method. The above advantage was confirmed by real image experiments.Development of lips image generation method based on multiple light sourceBy extending the method developed in (1) to a multiple light source system, we proposed a method for computing the reflectances of both a lips region and a skin region. As a result, it was shown that lips images from an arbitral vewpoint can be generated.

（1）开发从图像序列中恢复3-D LIPS形状的方法（2）我们开发了一种方法，从具有不确定量表的深度信息中提取绝对深度信息，这是通过Moire Imaging获得的，并具有简单的反射对象表面的模型。通过模拟和真实图像实验证实了它可以计算出无偏见的深度。（3）使用3-D Lips Shapewe提出了一种方法，可以计算出无偏见的深度。使用3-D Lips形状信息的元音识别和单词识别。提出的方法的优点是，可以完成针对面部和摄像机之间相对角的不变识别，并且通过应用提出的方法纯粹提高了识别率。实际图像实验证实了上述优势。基于多个光源的嘴唇图像生成方法的开发扩展到（1）中开发的方法到多光源系统，我们提出了一种计算嘴唇区域和嘴唇区域反射率的方法皮肤区域。结果，可以生成来自仲裁vewpoint的嘴唇图像。

项目成果

K.UDA, N.TAGAWA, A.MINAGAWA, T.MORIYA: "Efectiveness evaluation of speech feature including 3-D information for lipreading (in Japanese)"Proc. Of the 2000 Society Conference of IEICE. No.D-12-54. 241 (2000)

K.UDA、N.TAGAWA、A.MINAGAWA、T.MORIYA：“包括唇读 3D 信息在内的语音特征的有效性评估（日语）”Proc。

K.NIIKURA, A.MINAGAWA, N.TAGAWA, T.MORITA: "Determination of the absolute number of moirefringes using shading in formation"Proc.of Advances in Pattern Recoguition and Digital Techniques. 154-159 (1999)

K.NIIKURA、A.MINAGAWA、N.TAGAWA、T.MORITA：“使用形成中的阴影确定莫尔条纹的绝对数量”Proc.of Advances in Pattern Recoguition and Digital Techniques。

浅倉, 皆川, 田川, 守屋: "陰影情報を用いたモアレ縞の絶対縞次数決定法"電子情報通信学会技術研究報告. IE-99-36. 81-88 (1999)

Asakura、Minakawa、Takawa、Moriya：“使用阴影信息的莫尔条纹的绝对条纹顺序确定方法”IEICE 技术报告。

S.NAGANUMA, A.MINAGAWA, N.TAGAWA, T.MORIYA: "Computation of 3-D shape and reflectance with shading model and moire phase (in Japanese)"Proc. Of the 2001 IEICE General Conference. No.D-12-67. 234 (2001)

S.NAGANUMA、A.MINAGAWA、N.TAGAWA、T.MORIYA：“使用阴影模型和莫尔相位计算 3-D 形状和反射率（日语）”Proc。

A.INAGAKI, N.TAGAWA, A.MINAGAWA, T.MORIYA: "Computation of shape and reflectance of 3-D object using moire phase and reflectance model (in Japanese)"Technical report of IEICE. No.PRMU 2000-229. 51-56 (2001)

A.INAGAKI、N.TAGAWA、A.MINAGAWA、T.MORIYA：“使用莫尔相位和反射率模型计算 3-D 物体的形状和反射率（日语）”IEICE 的技术报告。