Multimodal, Negotiated Interaction in Mobile Scenarios

移动场景中的多模式协商交互

基本信息

批准号：
EP/E042740/1
负责人：
Roderick Murray-Smith
金额：
$ 50.6万
依托单位：
University of Glasgow
依托单位国家：
英国
项目类别：
Research Grant
财政年份：
2007
资助国家：
英国
起止时间：
2007 至无数据
项目状态：
已结题

来源：
https://gtr.ukri.org/projects?ref=EP%2FE042740%2F1
关键词：
Multimodal Negotiated Interaction Mobile Scenarios

项目摘要

Our proposal is to investigate an alternative means of allowing users to interact with content and services in their environment such that the actions they make, movements, gestures, etc., and feedback they receive are continuous, with the user and system negotiating their interactions in a fluid, dynamic way. We believe the appropriate comparison would be dancing, rather than the current command & control metaphor. When someone dances with a partner there is a soft ebb and flow of control; sometimes one person leads, sometimes the other, this changing fluidly as they dance. We are proposing a similar interaction between a user and computer, where sometimes the user leads and at other times the computer according to the context of the interaction. This contrasts with most current approaches where one agent, be it the human or the computer, pre-empts the other and where most interaction is driven by events and proceeds to varying degrees in rigid, over specified waysExample scenario: Exploring a Digitally Enriched EnvironmentAn example of how this approach could be used is that of location-aware information acquisition while walking in a town centre. You might feel a 'tick' on your phone's vibration motor, making you aware that there is information available about something in your environment. Your rich context understanding abilities would tell you how likely this 'tick' was to be of interest, if you ignore the cue and walk on, the negotiation would end there and then. If you are curious, you might gesture with the phone at likely targets in your surroundings, and get a response from several of them. If you are further intrigued, you may continue to interact with these potential targets, possibly moving from the vibro-tactile to an audio display, gaining information by an active exploration of the environment, something we have evolved to do naturally. The user explores the possibilities in the situation by directly engaging (probing or playing) with it, being able to move at will through the space of possibilities, gaining more and more insight during the interaction. The multimodal feedback provided both encodes the system's current interpretation of the user's intention (e.g. moving towards a target) and the probability of the target meeting the user's needs. After working through combinations of vibration and audio, if the joint dynamics of information source, and user continue to intertwine, the display of the mobile device might be used for full details. This example shows a 'schedule' of modalities, and illustrates the negotiation process in a practical and commercially interesting example.

我们的建议是调查允许用户在其环境中与内容和服务互动的另一种方法，以使他们所做的动作，动作，手势等以及他们收到的反馈是连续的，用户和系统以流畅的，动态的方式谈判他们的交互。我们认为，适当的比较将是跳舞，而不是当前的命令和控制隐喻。当某人与伴侣跳舞时，会有柔和的振奋和控制流动。有时，一个人在跳舞时会导致这种情况变化。我们提出了用户和计算机之间的类似交互，有时用户会根据交互的上下文领导计算机。这与大多数当前的方法形成鲜明对比的是，一种代理，无论是人类还是计算机，都会抢占另一个代理，并且大多数互动是由事件驱动的，并且在刚性的情况下以不同程度的方式驱动，而不是特定的witeSexample场景：探索数字丰富的环境的示例，即如何使用这种方法，是在居住在城市中心中的位置宣布信息的范围。您可能会在手机的振动电机上感到“ tick”，使您意识到环境中有有关某些内容的信息。您丰富的上下文理解能力将告诉您这种“ tick”有多大可能引起人们的兴趣，如果您忽略了提示和行走，谈判将结束。如果您很好奇，您可能会以手机在周围的可能目标上进行手势，并从其中几个响应中得到回应。如果您受到进一步的兴趣，您可能会继续与这些潜在目标相互作用，可能会从氛围转变为音频显示，并通过对环境的积极探索获得信息，这是我们已经发展为自然而然的。用户通过直接参与（探测或播放）来探讨情况中的可能性，能够随意通过可能性的空间移动，在互动过程中获得越来越多的见解。多模式反馈提供了既编码系统对用户意图的当前解释（例如，朝着目标迈进）以及目标的可能性满足用户需求。通过振动和音频的组合工作，如果信息源的联合动力和用户继续交织在一起，则可以将显示移动设备的显示用于完整的详细信息。此示例显示了模式的“时间表”，并在实用且商业上有趣的示例中说明了谈判过程。