HRI: Perceptually Situated Human-Robot Dialog Models

HRI：感知情境人机对话模型

• 批准号: 0704479
• 负责人: Trevor Darrell
• 金额: $ 81.5万
• 依托单位: Massachusetts Institute of Technology
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2007
• 资助国家: 美国
• 起止时间: 2007-09-01 至 2008-02-29
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0704479&HistoricalAwards=false
• 关键词: HRI; Perceptually; Situated; Human; Robot

Humans naturally use dialog and gestures to discuss complex phenomena and plans, especially when they refer to physical aspects of the environment while they communicate with each other. Existing robot vision systems can sense people and the environment, but are limited in their ability to detect the detailed conversational cues people often rely upon (such as head pose, eye gaze, and body gestures), and to exploit those cues in multimodal conversational dialog. Recent advances in computer vision have made it possible to track such detailed cues. Robots can use passive measures to sense the presence of people, estimate their focus of attention and body pose, and to recognize human gestures and identify physical references. But they have had limited means of integrating such information into models of natural language; heretofore, they have used dialog models for specific domains and/or were limited to one-on-one interaction. Separately, recent advances in natural language processing have led to dialog models that can track relatively free-form conversation among multiple participants, and extract meaningful semantics about people's intentions and actions. These multi-party dialog models have been used in meeting environments and other domains. In this project, the PI and his team will fuse these two lines of research to achieve a perceptually situated, natural conversation model that robots can use to interact multimodally with people. They will develop a reasonably generic dialog model that allows a situated agent to track the dialog around it, know when it is being addressed, and take direction from a human operator regarding where it should find or place various objects, what it should look for in the environment, and which individuals it should attend to, follow, or obey. Project outcomes will extend existing dialog management techniques to a more general theory of interaction management, and will also extend current state-of-the-art vision research to be able to recognize the subtleties of nonverbal conversational cues, as well as methods for integrating those cues with ongoing dialog interpretation and interaction with the world.Broader Impacts: There are clearly many positive societal impacts that will derive from this research. Ultimately, development of effective human-robot interfaces will allow greater deployment of robots to perform dangerous tasks that humans would otherwise have to perform, and will also enable greater use of robots for service tasks in domestic environments. As part of the project, the PI will conduct outreach efforts to engage secondary-school students in the hope that exposure to HRI research may increase their interest in science and engineering studies.

人类自然地使用对话和手势来讨论复杂的现象和计划，尤其是当他们在相互交流时提到环境的物理方面时。 现有的机器人视觉系统可以感知人和环境，但在检测人们经常依赖的详细对话线索（例如头部姿势、眼睛注视和身体姿势）以及在多模式对话对话中利用这些线索的能力有限。 计算机视觉的最新进展使得追踪此类详细线索成为可能。 机器人可以使用被动措施来感知人的存在，估计他们的注意力焦点和身体姿势，并识别人类手势并识别物理参考。 但他们将这些信息整合到自然语言模型中的方法有限。迄今为止，他们已经使用了特定领域的对话模型和/或仅限于一对一的交互。 另外，自然语言处理的最新进展催生了对话模型，可以跟踪多个参与者之间相对自由形式的对话，并提取有关人们意图和行为的有意义的语义。 这些多方对话模型已用于会议环境和其他领域。 在这个项目中，PI 和他的团队将融合这两条研究路线，以实现一个感知定位的自然对话模型，机器人可以使用该模型与人类进行多模式交互。 他们将开发一个相当通用的对话模型，允许定位代理跟踪周围的对话，了解对话何时被处理，并接受人类操作员的指导，了解应该在哪里找到或放置各种对象，应该在其中寻找什么环境，以及它应该关注、跟随或服从哪些个体。 项目成果将把现有的对话管理技术扩展到更通用的交互管理理论，还将扩展当前最先进的视觉研究，以便能够识别非语言对话线索的微妙之处，以及整合这些线索的方法持续的对话解释和与世界的互动的线索。更广泛的影响：这项研究显然会产生许多积极的社会影响。 最终，有效的人机界面的开发将允许更多地部署机器人来执行人类本来必须执行的危险任务，并且还将允许更多地使用机器人在家庭环境中执行服务任务。 作为该项目的一部分，PI 将开展外展活动，吸引中学生参与，希望接触 HRI 研究可以增加他们对科学和工程研究的兴趣。