RI: Small: RUI: Benchmarks and Algorithms for Mobile Image Matching

RI：小型：RUI：移动图像匹配的基准和算法

基本信息

批准号：
1718376
负责人：
Daniel Scharstein
金额：
$ 30万
依托单位：
Middlebury College
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2017
资助国家：
美国
起止时间：
2017-08-01 至 2021-07-31
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1718376&HistoricalAwards=false
关键词：
RI Small RUI Benchmarks Algorithms

项目摘要

This project will provide both new benchmarks and new algorithms for the mobile image matching domain. In order to drive and focus new research on mobile image matching, the existing Middlebury benchmarks will be augmented with new datasets of calibrated multiview and video sequences acquired with mobile devices, together with ground-truth geometry. The project will also contribute novel algorithmic approaches for robust and scalable image matching. Undergraduate students will be actively involved in all components of this project, in particular in the data acquisition and testing stages, as well as the authoring of online evaluation tools. The project will have several broader impacts. First, challenging benchmarks will serve as catalysts for new research. High-quality datasets are also useful beyond benchmarking in that they can aid algorithm design and enable learning approaches. Second, scalable and robust matching techniques tailored for mobile devices will enable a host of new applications with broad impacts on the population at large, including interactive 3D modeling of objects and people for social media, online commerce, and augmented and virtual reality. Finally, the project will expose undergraduates at a liberal-arts college in rural Vermont to the world of research, experimentation, and discovery.This project will augment the existing Middlebury datasets with calibrated multi-view and video sequences acquired with mobile devices from a robot arm, of challenging scenes with known geometry, derived using structured lighting. Datasets will include IMU data and flash/no-flash image pairs. The project will also explore novel evaluation metrics as well as the utilization of high-quality synthetic image sequences. A subset of the new datasets will be employed in new benchmarks for mobile 3D reconstructions tasks. The algorithmic work will contribute novel approaches for robust and scalable image matching. While the current trend in the community is to learn general models from large sets of labeled training data, this project will instead aim to learn data terms from the images at hand during the matching process. Such self-adjusting data terms will model radiometric and geometric distortions rather than being invariant to them. Another focus will be on memory-efficient approaches that avoid an exhaustive search of the full matching space while explicitly reasoning about occlusion, reflections, and transparency. Additional algorithmic techniques will include layer-based image matching algorithms, novel smoothness terms suitable for fast and scalable image matching, and novel strategies for dealing with completely textureless scenes.

该项目将为移动图像匹配领域提供新的基准和新算法。为了推动和集中对移动图像匹配的新研究，现有的 Middlebury 基准将通过移动设备获取的校准多视图和视频序列的新数据集以及地面实况几何进行增强。该项目还将为稳健且可扩展的图像匹配提供新颖的算法方法。本科生将积极参与该项目的所有组成部分，特别是在数据采集和测试阶段，以及在线评估工具的编写。该项目将产生几个更广泛的影响。首先，具有挑战性的基准将成为新研究的催化剂。高质量数据集除了基准测试之外也很有用，因为它们可以帮助算法设计并支持学习方法。其次，为移动设备量身定制的可扩展且强大的匹配技术将催生一系列对广大人群产生广泛影响的新应用，包括用于社交媒体、在线商务以及增强和虚拟现实的物体和人的交互式 3D 建模。最后，该项目将使佛蒙特州乡村一所文理学院的本科生接触研究、实验和发现的世界。该项目将通过机器人移动设备获取的校准多视图和视频序列来增强现有的 Middlebury 数据集手臂，具有已知几何形状的挑战性场景，使用结构照明导出。数据集将包括 IMU 数据和闪光/非闪光图像对。该项目还将探索新颖的评估指标以及高质量合成图像序列的利用。新数据集的子集将用于移动 3D 重建任务的新基准。该算法工作将为稳健且可扩展的图像匹配提供新方法。虽然社区当前的趋势是从大量标记的训练数据中学习通用模型，但该项目的目标是在匹配过程中从手头的图像中学习数据项。这种自我调整的数据项将对辐射和几何畸变进行建模，而不是对它们保持不变。另一个重点是内存有效的方法，避免对完整匹配空间进行详尽的搜索，同时明确地推理遮挡、反射和透明度。其他算法技术将包括基于图层的图像匹配算法、适合快速和可扩展图像匹配的新颖平滑项，以及处理完全无纹理场景的新颖策略。