Enabling rich, open-ended human-robot interaction through robust, advanced multimodal perceptual capabilities for high-level reasoning

通过强大、先进的多模态感知能力进行高级推理，实现丰富、开放式的人机交互

• 批准号: RGPIN-2019-06047
• 负责人: Ferland, François
• 金额: $ 1.68万
• 依托单位: Université de Sherbrooke
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2020
• 资助国家: 加拿大
• 起止时间: 2020-01-01 至 2021-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=713420
• 关键词: Enabling; rich; open; ended; human

The ultimate goal in Human-Robot Interaction is to have autonomous robots that can naturally interact with people, assist them in their daily lives at home or at work, and automatically adapt to new situations. Building such robots brings a great number of scientific challenges: safe, natural and effective interaction implies advanced perceptual capabilities to supply information for sound and robust reasoning. As humans operate in dynamic environments, robots will need a broad understanding of the world we live in and reasoning capabilities outside of the usually rigid, pre-defined scenarios they are currently programmed for. My research program is oriented toward enabling rich, open-ended interactions between humans and autonomous mobile robots. Doing so require high-level reasoning, which I believe is only possible with large-scale and multimodal perceptual capabilities on robots. To provide such capabilities, and as identified by studies on selective attention (SA) in neurosciences, there is growing evidence that top-down modulation occurs in the brain to maintain performance of the working memory by prioritizing already encoded items depending on the changing tasks demands and expectations of dynamic environments. A robot meant for rich interaction will have to adapt itself to such environments and prioritize what needs to be encoded in its own working memory. Furthermore, it will have to anticipate perceptual requirements from long-term memories of past experiences and manage its computing resources for future expected stimuli. To study such mechanisms, I will build a framework for distributed perceptual processing at a large scale based on existing open-source technologies to combine the stimuli of multiple stationary and mobile robot-embedded sensors and balancing the resources of multiple computing systems. This will be done to investigate how a robot can build multimodal understandings of their surroundings. Then, an SA-inspired mechanism will also filter the percepts encoded in both working and long-term memories to avoid overloading computing resources. Finally, to go beyond reactive resource management and prepare for expected future events, an anticipatory supervision mechanism will be designed to infer which stimuli to expect from past experiences. Validation of this research program will be done with wheeled humanoid robots with advanced manipulation and sensing capabilities in realistic settings. Participating to an international competition such as RoboCup@Home is planned to serve as common ground for evaluation of the added value of the mechanisms developed in relation to state-of-the-art in the field. This research program represents a unique opportunity to study embodied artificial intelligence (AI) to respond and anticipate to events in real interactive environments with humans. The program will include multiple projects for MSc and PhD students to train them as experts in applied AI and robotics.

人类机器人互动的最终目标是拥有可以自然与人互动的自动机器人，帮助他们在家中或工作中的日常生活，并自动适应新的情况。建造这样的机器人带来了许多科学挑战：安全，自然和有效的互动意味着先进的感知能力，以提供信息和稳健推理的信息。当人类在动态环境中运作时，机器人将需要对我们所生活的世界和推理能力的广泛理解，而在他们当前编程的通常僵化，预定的场景之外。 我的研究计划面向实现人类与自动移动机器人之间的丰富，开放式的互动。这样做需要高水平的推理，我认为只有在机器人上具有大规模和多模式的感知能力，才有可能。为了提供此类功能，并通过对神经科学的选择性注意（SA）的研究确定，越来越多的证据表明，自上而下的调制发生在大脑中发生，以维持工作记忆的性能，通过优先针对已编码的项目来确定任务的优先级，具体取决于不断变化的任务需求和对动态环境的期望。用于丰富互动的机器人必须适应此类环境，并确定需要在其自己的工作记忆中编码的内容。此外，它将必须从过去经验的长期记忆中预期感知要求，并管理其计算资源来实现未来的预期刺激。 为了研究这种机制，我将基于现有的开源技术建立一个大规模分布式感知处理的框架，以结合多个固定和移动机器人安装的传感器的刺激，并平衡多个计算系统的资源。这将是为了调查机器人如何建立对周围环境的多模式理解。然后，一种受SA启发的机制还将滤除工作和长期记忆中编码的感知，以避免计算资源过载。最后，为了超越反应性资源管理并为预期的未来事件做准备，将设计一种预期的监督机制，以推断出从过去的经验中期望哪些刺激。 该研究计划的验证将使用具有先进的操纵和感应能力的车轮类人体机器人进行。计划参加诸如Robocup@Home之类的国际竞争，以作为评估与该领域最新技术相关的机制附加价值的共同点。 该研究计划代表了研究体现人工智能（AI）的独特机会，以对与人类的真实互动环境中的事件做出反应和预期。该计划将包括MSC和PhD学生的多个项目，以作为Applied AI和机器人技术专家培训他们。