Assessing the validity of facilitated-volunteered geographic information: comparisons of expert and novice ratings

Facilitated-voluntary geographic information (f-VGI) is a promising method to enable systematic collection of data from residents about their physical and social environment. The method capitalizes on ubiquitous mobile smartphones to empower collection of geospatially-referenced data. It is important to evaluate the validity of user-generated content for use in research or program planning. The purpose of this study was to test whether the aggregated environmental (“bikeability”) ratings from novice community residents converges with ratings from experts using a robust research-based, paper audit-tool (the established Pedestrian Environment Data Scan (PEDS) tool). Equivalence testing statistically showed overall agreement between the composite ratings of bikeability within the novice group. Agreement in categorical ratings between novices and experts were examined using the summary agreement index, which showed substantial agreement across the 10 locations rated by 11 novices using an f-VGI mobile application and four experts using PEDS; variability depended on the nature of the specific questions asked. Results reveal overall substantial agreement between novice and expert ratings for both composite scores and individual categorical ratings. However, additional research is needed to refine the methodology for use in formalized research applications.

辅助式自愿地理信息（f - VGI）是一种很有前景的方法，能够从居民那里系统地收集有关其物理和社会环境的数据。该方法利用无处不在的移动智能手机来实现对地理空间参考数据的收集。评估用户生成内容在研究或项目规划中的有效性是很重要的。本研究的目的是测试新手社区居民对环境（“骑行适宜性”）的综合评分是否与专家使用一种基于严谨研究的纸质审核工具（已确立的行人环境数据扫描（PEDS）工具）的评分一致。等效性检验从统计学上表明新手组内骑行适宜性综合评分之间总体一致。使用综合一致性指数检验了新手和专家在分类评分上的一致性，结果显示11名新手使用f - VGI移动应用程序和4名专家使用PEDS对10个地点进行评分时，总体上具有高度一致性；差异取决于所提具体问题的性质。结果显示，在综合评分和单个分类评分方面，新手和专家的评分总体上高度一致。然而，还需要进一步的研究来完善该方法，以便用于正式的研究应用。