A Study on Usages of Non-verbal Information for a Design of Information Assistive Tools

非语言信息在信息辅助工具设计中的使用研究

基本信息

批准号：
14208028
负责人：
IFUKUBE Tohru
金额：
$ 25.04万
依托单位：
The University of Tokyo
依托单位国家：
日本
项目类别：
Grant-in-Aid for Scientific Research (A)
财政年份：
2002
资助国家：
日本
起止时间：
2002 至 2004
项目状态：
已结题

来源：
https://kaken.nii.ac.jp/grant/KAKENHI-PROJECT-14208028/
关键词：
barrier free non-verbal information the hearing impaired automatic captioning system facial expression the visually impaired screen reader tactile information 音声認識タクタルエイド復唱字幕タクタイルエイド

项目摘要

We investigated how non-verbal information should be utilized in order to improve two information acquisition tools which we designed for the visually and auditory impaired.1.One of the tools is a new screen reader system which converts texts on the screen into synthesized speech while displaying rich text information, i.e. non-verbal information, onto a finger-tip surface by using a tactile jog-dial (TAJODA). This is a supporting tool of a web-information acquisition for visually impaired people.2.The other tool is a new hearing assistive device (captioning system) which automatically converts speech into texts while displaying speaker's lip-movement and facial expression, i.e. non-verbal information, for the acquired hearing impaired.Research results regarding "the screen reader" are as followings.(1)The speed of synthesized speech should be changed up to 3 times faster than normal speaking rate since our findings showed the maximum listening speeds for the blind is 2.6 times faster … More than the normal persons.(2)The optimal number of rich texts to transmit is 7;i.e. font sizes (larger or smaller than the standard), colored letter, bold, parentheses (left or right), and paragraph.(3)Experimental results by 8 blind users showed that the acquisition speed of web sentences becomes 2.6 times faster than ordinal screen reader.Research results regarding "the captioning system" are as followings.(1)We investigated how a speaker's facial image and lip-movement (the non-verbal information) were useful for the comprehension of the sentenses(2)Experimental results show that the lip-movement was more effective than facial expression and about 2 sec delay of the lip-movement display from the captions was suitable for the fascilitation of the comprehension.(3)By combining this captioning system with the non-verbal information, the comprehension was proved apparently fascilitated through about 30 times trials at international conferences.Non-verbal information ; the tactile stimulation in TAJODA and the lip-movement in the captioning system was apparently effective to fascilitate the comprehension of contents. Furthermore, we could determin the optimal parameters of the display timing and the display method of the non-verbal information in order to improve the information assistive tools for the sensory disabled. Less

我们研究了如何利用非语言信息，以改进我们为视觉和听觉障碍人士设计的两种信息获取工具。1.其中一个工具是一种新的屏幕阅读器系统，它将屏幕上的文本转换为合成语音，同时将屏幕上的文本转换为合成语音。通过使用触觉滚轮（TAJODA）将丰富的文本信息（即非语言信息）显示到指尖表面上。这是视觉障碍人士获取网络信息的支持工具。2.另一个工具是。一个新型听力辅助设备（字幕系统），可自动将语音转换为文本，同时为后天性听力障碍者显示说话者的嘴唇动作和面部表情，即非语言信息。有关“屏幕阅读器”的研究结果如下。（1 ）合成语音的速度应比正常说话速度快 3 倍，因为我们的研究结果表明盲人的最大听力速度比正常人快 2.6 倍……更多。（2）最佳数量传输的富文本为7；即字体大小（大于或小于标准）、彩色字母、粗体、括号（左或右）、段落。（3）8位盲人用户的实验结果表明，网页的获取速度比普通屏幕阅读器快2.6倍。关于“字幕系统”的研究结果如下。（1）我们研究了说话者的面部图像和嘴唇运动（非语言信息）对句子的理解有帮助(2)实验结果表明，唇动比面部表情更有效，并且字幕中唇动显示延迟约2秒适合促进理解。(3 ）通过将这种字幕系统与非语言信息相结合，经过大约30次国际会议的试验，证明理解效果明显增强。 TAJODA和字幕系统中的口型运动对于促进内容的理解显然是有效的，此外，我们还可以确定非语言信息的显示时机和显示方式的最佳参数，以改进信息辅助工具。对于感觉障碍者来说较少。