Robot arm and hand manipulation using vision-guided motion control directed by human gestures

使用由人类手势指导的视觉引导运动控制的机器人手臂和手部操作

• 批准号: RGPIN-2016-06365
• 负责人: Jagersand, Martin
• 金额: $ 3.93万
• 依托单位: University of Alberta
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2019
• 资助国家: 加拿大
• 起止时间: 2019-01-01 至 2020-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=690470
• 关键词: Robot; arm; hand; manipulation; using

This research will focus on the geometric specification of robot (arm and hand) manipulation tasks in camera (and other sensory) coordinate frames. We propose to develop Human-Robot Interfaces (HRI) based on pointing and gestures. The gestures will be interpreted through computer vision and robot motions will be performed through visual servoing, and force servoing. The main objective is to solve general robot arm and hand manipulation tasks interactively with humans.******Robotics research in both engineering and artificial intelligence has focused on autonomy. While autonomy works well in specific situations (such as factory automation), despite decades of research, robotics cannot solve human arm and hand manipulation tasks in everyday unstructured environments. However, in many cases autonomy may not be the goal. Humans naturally interact and collaborate on tasks. A human can easily guide another human how to perform complex tasks. Much of this interaction is non-verbal. For robots to work with humans they need to take visual cues from the human, and be easy to interact with, rather than working autonomously. Hence, with good HRI, humans could relatively effortlessly supervise tasks that are otherwise too complex for robots. ******We propose an integrated systems and experiment-driven approach to this research. In particular we will research:******1. Task specification. We will develop the so called "task-function" to handle a combination of camera image-, 3D world- and force sensor coordinates.***2. Video tracking of image features. We will research how to improve robustness of registration-based tracking.***3. Hybrid Visual Servoing. We will extend image-coordinate visual servoing to also work with 3D and force sensory feedback, while requiring a minimum of a-priori calibration.***4. HRI. We will research gesture based human-robot interaction, including what geometric specifications (2D image, 3D, force contacts) are most natural for a human in order to specify complete, provable 3D alignments; and then to sequence into full manipulation tasks. We will develop benchmarks for everyday human manipulation tasks, and study our task specification interfaces on these (e.g. task completion times and cognitive load.)******The goal of our research will be a better understanding of visual and sensory-space motion control, both from a robotics and human interface perspective. Practical outcomes include integration of different tracking and sensory-motor control methods into modular ROS (Robot Operating System) software libraries, and test results of this software by humans on many different manipulation tasks.

这项研究将重点关注相机（和其他感官）坐标系中机器人（手臂和手）操纵任务的几何规范。我们建议开发基于指向和手势的人机界面（HRI）。手势将通过计算机视觉进行解释，机器人运动将通过视觉伺服和力伺服来执行。主要目标是解决与人类交互的一般机器人手臂和手部操作任务。******工程和人工智能领域的机器人研究都集中在自主性上。虽然自主性在特定情况下（例如工厂自动化）运作良好，但尽管经过数十年的研究，机器人技术仍无法解决日常非结构化环境中的人类手臂和手部操作任务。然而，在许多情况下，自治可能不是目标。人类自然地在任务上互动和协作。一个人可以轻松地指导另一个人如何执行复杂的任务。这种互动大部分是非语言的。对于与人类一起工作的机器人来说，它们需要从人类那里获取视觉线索，并且易于与之交互，而不是自主工作。因此，有了良好的 HRI，人类就可以相对轻松地监督那些对于机器人来说过于复杂的任务。 ******我们提出了一种集成系统和实验驱动的方法来进行这项研究。我们将特别研究：*****1。任务规范。我们将开发所谓的“任务功能”来处理相机图像、3D 世界和力传感器坐标的组合。***2。图像特征的视频跟踪。我们将研究如何提高基于注册的跟踪的稳健性。***3。混合视觉服务。我们将扩展图像坐标视觉伺服，使其也可以与 3D 和力感觉反馈一起使用，同时需要最少的先验校准。***4。人权研究所。我们将研究基于手势的人机交互，包括哪些几何规范（2D 图像、3D、力接触）对于人类来说是最自然的，以便指定完整的、可证明的 3D 对齐；然后排序为完整的操作任务。我们将为日常人类操作任务制定基准，并研究我们的任务规范接口（例如任务完成时间和认知负荷）。 *****我们研究的目标将是更好地理解视觉和感官空间运动控制，无论是从机器人技术还是人机界面的角度来看。实际成果包括将不同的跟踪和感觉运动控制方法集成到模块化 ROS（机器人操作系统）软件库中，以及人类在许多不同的操作任务上测试该软件的结果。