Research on an effective method of video communication of sign language in consideration of cognitive characteristics

考虑认知特征的有效手语视频交流方法研究

基本信息

批准号：
16091209
负责人：
NAGASHIMA Yuji
金额：
$ 11.01万
依托单位：
Kogakuin University
依托单位国家：
日本
项目类别：
Grant-in-Aid for Scientific Research on Priority Areas
财政年份：
2004
资助国家：
日本
起止时间：
2004 至 2006
项目状态：
已结题

来源：
https://kaken.nii.ac.jp/en/grant/KAKENHI-PROJECT-16091209/
关键词：
Cognitive Science Sign Linguistic Engineering Video Communication ERP ICT on Well-being Corpus Commnnication Assistive Technology 画像評価動画像符号化

项目摘要

This research has aimed at the communication support using ICT for the hearing impaired people. Provided result is the following items.(a) We are concerned with language cognition, semantic processing process analysis based on N400 ; event related potential measurement was examined for the understanding the Japanese sign language comprehension mechanisms. As a result, because it is understood that N400 is observed in semantic congruity sentence by sign language and the character, and the potential is large in the character stimulation, and was shown the difference of the processing process by the presentation modality.(b) We discussed evaluation techniques for the quality of digitally encoded motion picture of sign language. This result guarantees that opinion test can be substituted for intelligibility test. Further investigation shows that motion pictures with high frame rate are more legible than those with high precision. Another result shows that screen size of sign picture doesn' … More t affect comprehensibility.(c) We are concerned with low-bit rate digital encoding of moving picture of sign language. A new encoding method suitable for sign language is presented based on characteristics of sign language. As a result, an encoding image data using our proposed method is more possible at a low bit rate in comparison with an existing method. Moreover, our method in the same parameter condition has been confirmed that it was obtained a clearer image in comparison with an existing method.(d) We have also worked on a problem of mental workload, which is mental stress that sign language users undergo during interactive communication. We have found that NASA-TLX is effective enough for measuring the degree of mental workload(e) In the research process, we collected a corpus by video image and electronic data by three-dimensional movement for linguistic analysis of sign language. The electronic data showed it on Web as sign language animation by the BoT3D engine.Our findings have been appeared in journals such as the one of the transactions on IEICE, and the one of Human Interface Society.We feel necessity to develop this research. On the basis of result of this research, we continue analysis work from the side of cognitive science through conversation analysis of sign language. Less

这项研究旨在使用听力障碍的人，以语言认知，语义处理过程低音角色和特征刺激的潜力很大，并且通过处理方式（b）来显示处理方式的差异可以替代智能测试，而不是高精度的数字编码。此外，使用比特率的甲基苯丙胺与现有方法进行了比较，我们在相同的参数条件下使用的方法已经确定了它与现有方法相比，它也获得了更清晰的方法。在互动沟通期间，我们发现NASA-TLX足以衡量研究过程中的心理工作量（E），我们通过视频图像和电子数据通过三辅助秘服收集了语料库，以便对语言进行语言分析。电子数据在Web上显示为Bot3D引擎的手语动画。我们的发现与IEICE上的交易相同。通过对语言的对话分析