Ground truth inference and quality control of geospatial data collection by paid crowdworkers for the efficient acquisition of training data for deep learning systems

由付费众包人员对地理空间数据收集进行地面实况推理和质量控制，以有效获取深度学习系统的训练数据

• 批准号: 501973633
• 负责人: Professor Dr. Uwe Sörgel
• 金额: --
• 依托单位: Institut für Photogrammetrie
• 依托单位国家: 德国
• 项目类别: Research Grants
• 资助国家: 德国
• 项目状态: 未结题
• 来源: https://gepris.dfg.de/gepris/projekt/501973633?language=en
• 关键词: Ground; truth; inference; quality; control

Currently, great efforts are being made to apply Deep Learning systems like Convolutional Neural Networks (CNN) also to remote sensing images. However, due to the peculiarities of remote sensing images, standard CNNs are of limited use for their analysis. It would be desirable to train specialized CNNs from scratch, but this is yet not possible due to the lack of the required large amount of annotated training data.Crowdsourcing offers an effective method for providing such data, which has led to increasing interest in using this method to collect geospatial data from remote sensing images. However, the crowd is composed of people with very different backgrounds, most of whom are not familiar with geospatial data collection standards. Therefore, we must expect results of very heterogeneous quality. The objective of this project is to enable the collection of high-quality data from remote sensing images by paid crowdworkers. The process is designed such that no time-consuming manual inspection of the results is necessary even if no reference data are available. We suggest a data-driven approach based on multiple data collection to describe and improve the geometric quality of the collected data. First, we define an integrated quality measure that quantifies the similarity of two geometric representations (one collected by a crowdworker, one the corresponding ground truth) of a geographic object with one numerical value. We will derive this measure based on statistical evaluations by using an approach from the information theory. As next step, we will integrate multiple representations into one common geometry. We will use the quality measure on the one hand to evaluate the quality of the integrated geometries and on the other hand to optimize the integration process. This can be realized even intrinsically without comparison to given ground truth.Then, we want to investigate if by using a CNN an automated quality evaluation can be realized also without multiple data collection. The input of this CNN will be a remote sensing image and one individual geometry collected by a crowdworker. As output, the CNN shall predict a quality measure that describes how good the object was collected. Using such a CNN, we are able to avoid the necessity of multiple acquisitions of the same object. Consequently, collecting data will be much cheaper.In order to validate the generalizability of our approach, we apply it to scenes of quite different characteristics. This requires an adaptation of our model to different domains. For this purpose, we use an Active Learning approach, which iteratively performs domain adaptation in the interplay of crowdworkers and a CNN. Finally, we address possible follow-up research.

目前，人们正在努力将卷积神经网络（CNN）等深度学习系统应用于遥感图像。然而，由于遥感图像的特殊性，标准 CNN 在其分析中的用途有限。人们希望从头开始训练专门的 CNN，但由于缺乏所需的大量带注释的训练数据，这还不可能。众包提供了一种提供此类数据的有效方法，这导致人们对使用这种数据越来越感兴趣一种从遥感图像中收集地理空间数据的方法然而，人群由背景截然不同的人组成，其中大多数人不熟悉地理空间数据收集标准。因此，我们必须期待质量非常参差不齐的结果。 该项目的目标是让付费众包工作者能够从遥感图像中收集高质量数据。该过程的设计使得即使没有可用的参考数据，也不需要耗时的手动检查结果。我们建议采用基于多个数据收集的数据驱动方法来描述和提高所收集数据的几何质量。首先，我们定义了一种综合质量度量，用于量化具有一个数值的地理对象的两种几何表示（一种由众包收集，一种是相应的地面实况）的相似性。我们将使用信息论的方法根据统计评估得出这一衡量标准。下一步，我们将把多种表示集成到一个通用几何体中。我们将一方面使用质量测量来评估集成几何形状的质量，另一方面优化集成过程。这甚至可以在本质上实现，无需与给定的真实情况进行比较。然后，我们想要研究通过使用 CNN，是否可以在不收集多个数据的情况下实现自动质量评估。该 CNN 的输入将是一幅遥感图像和由众包人员收集的一个单独的几何图形。作为输出，CNN 应预测描述对象收集质量的质量度量。使用这样的 CNN，我们能够避免多次采集同一对象的必要性。因此，收集数据会便宜得多。为了验证我们的方法的普遍性，我们将其应用于具有完全不同特征的场景。这需要我们的模型适应不同的领域。为此，我们使用主动学习方法，该方法在众包工作者和 CNN 的相互作用中迭代地执行域适应。最后，我们讨论了可能的后续研究。