A Research on a Pattern Language Processing system for Human-Machine Interface

人机界面模式语言处理系统的研究

基本信息

批准号：
06808039
负责人：
YOSHIDA Takaichi
金额：
$ 1.22万
依托单位：
Kyushu Institute of Technology
依托单位国家：
日本
项目类别：
Grant-in-Aid for General Scientific Research (C)
财政年份：
1994
资助国家：
日本
起止时间：
1994 至 1995
项目状态：
已结题

项目摘要

We have built a basic environment which supports smooth communication between human and computers by gesture. At first, we aimed to build a pattern language processing system which make it possible to give instructions to computers by the finger's language. The system recognizes shapes and motion of fingers in a video camera image and interprets instructions which the finger's language intends to express. We designed a high speed algorithm which extracts finger shape from an image, and built the system as an experiment. This experiment afforded an opportunity for a new method substitutes for the finger language i.e.a method using a stick. This method has an advantage such as high speed processing and easiness to indicate a position of a stick.We found that an interface using a stick is more useful than the finger language, and we built a prototype interface system. The prototype interface system extracts a stick from a video camera image, and obtains the estimated three-dimensional coo … More rdinate value in real time. Because it became clear that a stick was able to be used as a three-dimensional pointing device, we built a multi-modal interface combined with speech recognition to examine operability. Then we implemented the three-dimensional block world as a prototype application and evaluated it. As a result, we found that the multi-modal interface is easy to use for users.At the same time, we carried on researches to make it possible to use the finger language as a pattern language in the future. To solve a problem of occlusion which is a main problem when we recognize pattern languages such as a finger language by using image processing, we examined a method using multi video cameras complementarily. Because speech recognition is necessary for a multi-modal interface, we examined a configuration of a speech recognition system and developed a new method which extracts human voice form background noise. According to its evaluation, we have conclude that the speech recognition system is useful even in a noisy environment. Less

我们已经建立了一个支持人与计算机之间通过手势进行顺畅通信的基础环境。首先，我们的目标是建立一个模式语言处理系统，使计算机可以通过手指的语言发出指令，系统可以识别手指的形状和动作。我们设计了一种从图像中提取手指形状的高速算法，并构建了该系统作为实验，该实验为替代该方法的新方法提供了机会。手指语言即使用棍子的方法。这种方法具有处理速度快、易于指示棍子位置等优点。我们发现使用棍子的界面比手指语言更有用，因此我们构建了一个原型界面系统。原型接口系统从摄像机图像中提取一根棍子，并实时获得估计的三维坐标值，因为很明显一根棍子可以用作三维指向设备。建立了多模式界面结合语音识别来检查可操作性，然后我们实现了三维块世界作为原型应用程序并对其进行了评估，结果发现多模态界面对于用户来说很容易使用。为了解决使用图像处理识别手指语言等模式语言时的主要问题——遮挡问题，我们进行了研究，以便将来可以使用手指语言作为模式语言。由于语音识别是必要的，因此需要使用多个摄像机互补的方法。通过多模态接口，我们检查了语音识别系统的配置，并开发了一种从背景噪声中提取人声的新方法，根据其评估，我们得出结论，该语音识别系统即使在嘈杂的环境中也是有用的。