ITR: Multimodal Human Computer Interaction: Toward a Proactive Computer

ITR：多模式人机交互：迈向主动式计算机

• 批准号: 0085980
• 负责人: Thomas Huang
• 金额: $ 315.21万
• 依托单位: University of Illinois at Urbana-Champaign
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2000
• 资助国家: 美国
• 起止时间: 2000-09-01 至 2006-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0085980&HistoricalAwards=false
• 关键词: ITR; Multimodal; Human; Computer; Interaction

This is the first year funding of a five-year continuing award. This project is based on the belief that, to be more accessible to the general population, computers must be more proactive in their interactions with people. In human interaction, someone who waits for each command before making any communication attempt would be regarded as uncooperative and unhelpful. In order for a computer to be more proactive and, thus, to bear its part of the burden of initiation in interactions, it must have (1) much more real-time information about its user, and (2) algorithms that select actions based on this information rather than simply on user commands. The computer needs information about the user's current and past emotional, motivational and cognitive state as well as the state of the task at hand. A theory, is needed to guide the development of algorithms that select appropriate actions based on user and task state. This research constitutes the next steps in an attempt by the PI's multidisciplinary team to develop this capability.Proposed research includes: (1) further development of methods to sense user postures, movements, expressions and speech; (2) analysis and fusion of this information to identify and track user states; (3) task state tracking; (4) creating a corpus of emotion- and action-labeled videotapes for use with computer learning; (5) further development of affective communication; (6) development of the basis for human-centered state-based action decisions; and (7) evaluation of computer proaction on human behavior and response. The testbed is an environment for hands-on education in science and engineering, using the Lego Mindstorms construction and robotics environment, with children of middle school age. An emphasis will be on developing proactive computing methods for encouraging interest and conceptual development of minority children and females, who often show lower achievement in science. Although the work will be conducted within an educational environment, the methods developed and studied will be broadly applicable, and this project should serve as an exemplar of the type of work that is needed in other computer-aided situations.

这是五年持续奖项的第一年资金。 该项目基于这样的信念，即，要使一般人群更容易获得计算机，计算机必须在与人的互动中更加主动。 在人类的互动中，一个等待每个命令之前进行任何交流尝试的人将被视为不合作和无济于事。 为了使计算机更加主动，因此承担了交互作用启动负担的一部分，它必须具有（1）有关其用户的更多实时信息，以及（2）基于此信息而不是简单地基于用户命令选择操作的算法。 计算机需要有关用户当前和过去的情感，动机和认知状态以及手头任务状态的信息。 需要一种理论来指导基于用户和任务状态选择适当动作的算法的开发。 这项研究构成了PI的多学科团队试图发展此能力的下一步。所提供的研究包括：（1）进一步开发方法，以感知用户姿势，运动，表达式和语音； （2）该信息的分析和融合以识别和跟踪用户状态； （3）任务状态跟踪； （4）创建与计算机学习一起使用的情感和动作标记的录像带； （5）进一步发展情感交流； （6）发展以人为本的国家行动决策的基础； （7）评估计算机对人类行为和反应的评估。 测试台是使用Lego Mindstorms的建筑和机器人技术环境，具有中学时代的孩子，是一个在科学和工程方面进行动手教育的环境。 重点是开发积极的计算方法，以鼓励少数族裔儿童和女性的兴趣和概念发展，这些方法通常在科学领域表现出较低的成就。 尽管这项工作将在教育环境中进行，但开发和研究的方法将广泛适用，该项目应作为其他计算机辅助情况下需要的工作类型的典范。