Fundamentals Studies about Recognition Mechanism Analysis for a Description Method of Sign language as Interaction Language

手语作为交互语言的描述方法的识别机制分析基础研究

基本信息

批准号：
14208030
负责人：
NAGASHIMA Yuji
金额：
$ 33.03万
依托单位：
KOGAKUIN UNIVERSITY
依托单位国家：
日本
项目类别：
Grant-in-Aid for Scientific Research (A)
财政年份：
2002
资助国家：
日本
起止时间：
2002 至 2005
项目状态：
已结题

项目摘要

A purpose of this research is scientifically clarify the mechanism of a linguistic function of a non-manual signals and a recognition function for communication of sign language. I got the following main result by this research through a research period.1.In analysis of mechanism of the recognition, I have analyzed a meaning processing process based on a event-related potential N400 measurement. As a result, because it is understood that N400 is observed in semantic congruity sentence by sign language and the character, and the potential is large in the character stimulation, and was shown the difference of the processing process by the presentation modality.2.In analysis support system, we have been constructing synchronous input and analysis system for video image and motion-captured data such as location and orientation of the hands and head in relation of the magnetic source. Simultaneously, we have been measuring hands configuration by gloved type sensors. As a result of using thi … More s system in actual analysis, it has solved the various kinds of problems that depended on visual inspection.3.In description method, it strengthened a description function of a non-finger signal of a hierarchical morpheme NVGS model of sign language.4.In analysis of conversation model, it was defining the lower limit of detection (LLD) of delay in sign conversation. Evaluation of the effect of delay was carried out by double stimulus impairment scale method (DSIS). The results showed that LLD of alternate counting was 100 msec, and LLD of "summing numbers" was 200 msec. Ceneralizing the results obtained so far, it is concluded that dialogue by sign language in more tolerant than speech dialogue.5.In low-bit rate digital encoding of moving picture of sign language, a new encoding method suitable for sign language is presented based on above mentioned characteristics of sign language. As a result, an encoding image data using our proposed method is more possible at a low bit rate in comparison with an existing method.This research purpose and the execution plan were almost achieved as above. In addition, results of research are presented in Journal and Transaction of IEICE, Journal and Transactions of Human Interface Society, Journal of IEE, International Conference, etc.I feel necessity to develop this research. On the basis of result of this research, I continue analysis work from the side of cognitive science through conversation analysis of sign language. Less

本研究的目的是科学地阐明非手工信号的语言功能和手语交流的识别功能的机制，经过一个时期的研究，得到了以下主要成果： 1、机制分析。在识别过程中，我分析了基于事件相关电位N400测量的意义处理过程，结果表明，N400是通过手语和字符在语义一致性句子中观察到的，并且潜力很大。人物刺激，并被展示2.在分析支持系统中，我们一直在构建视频图像和运动捕捉数据（例如手和头相对于磁力的位置和方向）的同步输入和分析系统同时，我们通过手套式传感器测量手的形状，在实际分析中使用该系统，解决了依赖目视检查的各种问题。3.在描述方法上。强化了描述功能4.在对话模型分析中,定义了手语对话中延迟的检测下限(LLD),并对延迟的影响进行了评价。双刺激损伤量表法（DSIS）的结果表明，交替计数的LLD为100毫秒，“求和数字”的LLD为200毫秒，总结目前获得的结果，得出结论是通过符号进行对话。 5、在手语运动图像低码率数字编码中，根据手语的上述特点，提出了一种适合手语的编码方法，得到了编码图像。与现有方法相比，使用我们提出的方法在低比特率下更有可能获得数据。本研究目的和执行计划基本达到了上述目的。此外，研究结果发表在《IEICE杂志》和《Transaction》、《Journal》和《Transaction》上。人机界面学会汇刊、IEE 杂志、国际会议、我觉得有必要在这项研究结果的基础上，通过手语对话分析继续进行认知科学方面的分析工作。