Deep Learning for Vision-based Measurement

基于视觉的测量的深度学习

基本信息

批准号：
RGPIN-2018-04405
负责人：
Lang, Jochen
金额：
$ 3.35万
依托单位：
University of Ottawa
依托单位国家：
加拿大
项目类别：
Discovery Grants Program - Individual
财政年份：
2022
资助国家：
加拿大
起止时间：
2022-01-01 至 2023-12-31
项目状态：
已结题

来源：
https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=752496
关键词：
Deep Learning Vision based Measurement

项目摘要

Deep learning techniques are remarkably successful on detection and recognition tasks in computer vision, reaching better than human performance in some specific applications. In my research I will develop novel learning-based algorithms and methods for virtual reality where data driven interaction and environment modelling, as well as user interfaces, can significantly benefit from advances in computer vision. Visual tracking allows different virtual reality and augmented reality devices to remain registered and synchronized. Motion estimation of surfaces, objects and actors is crucial in motion capture and 3D interaction modelling. Multi-view stereo captures real-word environments enabling navigation through these environment based on geometric relationships. My research program will significantly improve visual tracking, motion and stereo algorithms with learning-based techniques by focusing on vision based measurement. Vision based measurement uses the camera as a measurement instrument to obtain a measurand through an associated measurement procedure and with an uncertainty. This is crucial but often overlooked in virtual and augmented reality where different sensors and sensing results need to be fused with each other but also combined with physical reality. The geometry but also the appearance of captured objects and characters must not only appear realistic on their own but also when integrated into the whole virtual or augmented reality. In my research program, I will develop methods that on the one hand use physical constraints on the learning and on the other hand use learning to obtain physically plausible models. I will work on making consistent long-term tracking possible, develop real-time learning methods for motion estimation and 3D capture, thereby advancing the state-of-the-art in virtual and augmented reality through a focus on vision based measurement.

深度学习技术在计算机视觉中的检测和识别任务方面取得了非常成功的成功，在某些特定应用中的表现要比人类表现更好。在我的研究中，我将开发基于学习的新型算法和虚拟现实方法，在这些算法中，数据驱动的互动和环境建模以及用户界面可以从计算机视觉的进步中显着受益。视觉跟踪允许不同的虚拟现实和增强现实设备保持注册和同步。表面，物体和参与者的运动估计对于运动捕获和3D相互作用建模至关重要。多视图立体声捕获了基于几何关系通过这些环境导航的现实环境。我的研究计划将通过专注于基于视觉的测量来显着改善基于学习的技术的视觉跟踪，运动和立体声算法。基于视觉的测量将摄像机用作测量仪器，通过相关的测量程序和不确定性获得测量和测量工具。这是至关重要的，但在虚拟和增强现实中经常被忽略，在这些现实中，不同的传感器和感应结果需要相互融合，但也与物理现实结合在一起。几何形状，而且捕获的对象和角色的外观不仅必须自行出现现实，而且还必须集成到整个虚拟或增强现实中时。在我的研究计划中，我将开发一种方法，这些方法一方面使用对学习的物理约束，另一方面使用学习来获得物理上合理的模型。我将努力使长期跟踪成为可能，开发运动估算和3D捕获的实时学习方法，从而通过重点关注基于视觉的测量方法来推进虚拟和增强现实的最新现实。