Integrating Haptic and Visual Data for Low Latency transmission

集成触觉和视觉数据以实现低延迟传输

• 批准号: 2446549
• 金额: --
• 依托单位: King's College London
• 依托单位国家: 英国
• 项目类别: Studentship
• 财政年份: 2020
• 资助国家: 英国
• 起止时间: 2020 至 无数据
• 项目状态: 未结题
• 来源: https://gtr.ukri.org/projects?ref=studentship-2446549
• 关键词: Integrating; Haptic; Visual; Data; Low

Precisely determining the contact force during safe interaction in Minimally Invasive Surgery is still an open research challenge. Inspired by post-operative qualitative analysis from surgical videos, the use of cross-modality data driven deep neural network models has been one of the newest approach to predict sensorless force trends. The limited availability to vision-state-force datasets inspired the creation of a teleoperated laparoscopic forceps via robotic arm to recreate the forces of such tool when interacting with soft environments. To solve this task, we use a laparoscopic forceps mounted on a robotic arm, that we teleoperate using a haptic device. We collect a dataset containing visual, robot state and force readings for different types of soft silicone environments. We make a deep analysis on how different artificial intelligence models predict forces, first on a randomized training scheme. As we have a big variability on our dataset, we test the adaptability of these models to different geometries, colors and stiffnesses, and how good the visual and state vectors help on this part. Addtionally, we determined the effectiveness of such models to adapt and learn from multiple dataset.In the next part of my project, we decided to use generative deep learning techniques in order to model soft tissue dynamics from purely visual information. The approximation of such motion is done using modal coordinate analysis, really common in Finite Element Method simulations. We use the mathematical representation of the sinusoidal solutions on the frequency domain, using the Fast Fourier Transform, so we can further discretize the components and consider only a reasonable set of solutions.

精确确定微创手术中安全交互过程中的接触力仍然是一个开放的研究挑战。受手术视频术后定性分析的启发，使用跨模态数据驱动的深度神经网络模型已成为预测无传感器力趋势的最新方法之一。视觉状态力数据集的可用性有限，激发了通过机械臂创建遥控腹腔镜镊子的灵感，以在与软环境交互时重现此类工具的力。为了解决这项任务，我们使用安装在机械臂上的腹腔镜镊子，并使用触觉设备进行远程操作。我们收集了一个数据集，其中包含不同类型软硅胶环境的视觉、机器人状态和力读数。我们对不同的人工智能模型如何预测力进行了深入分析，首先是随机训练方案。由于我们的数据集有很大的可变性，我们测试这些模型对不同几何形状、颜色和刚度的适应性，以及视觉和状态向量在这方面的帮助程度。此外，我们确定了此类模型适应和学习多个数据集的有效性。在我的项目的下一部分中，我们决定使用生成深度学习技术来根据纯粹的视觉信息对软组织动力学进行建模。这种运动的近似是使用模态坐标分析完成的，这在有限元法模拟中非常常见。我们使用频域上正弦解的数学表示，使用快速傅里叶变换，因此我们可以进一步离散化分量并仅考虑一组合理的解。