CCRI: Planning: A Community-Standard, Large-Scale Synthetic 3D Scene Dataset for Scene Analysis and Synthesis

CCRI：规划：用于场景分析和合成的社区标准、大规模合成 3D 场景数据集

• 批准号: 2016532
• 负责人: Daniel Ritchie
• 金额: $ 5万
• 依托单位: Brown University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-10-01 至 2024-03-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2016532&HistoricalAwards=false
• 关键词: CCRI; Planning; Community; Standard; Large

To function as useful household assistants, robots need to understand what they are seeing and how to navigate in indoor environments. The current state-of-the-art approaches for solving these problems rely on machine learning, and in particular deep learning, which requires large quantities of labeled data (e.g. many images with per-pixel labels indicating what type of object is present at that pixel). Rather than asking people to laboriously label data captured from real-world spaces, a promising alternative approach is to use *synthetic* 3D scenes: virtual 3D models of indoor spaces. The 3D objects which populate these virtual spaces can be equipped with information such as their object type, which allows large sets of labeled training data to be created essentially “for free.” This project aims to construct *the* community-standard, large-scale synthetic 3D scene dataset. While some synthetic 3D scene datasets exist, they are either too small, or they have been subject to onerous use restrictions (and even lawsuits) due to copyright issues on their 3D models, which typically come from for-profit companies. This project will construct a large-scale dataset out of freely-available 3D content. The main contribution of the project is not just this dataset, but also a *scalable pipeline* for creating such 3D scene datasets. This pipeline will be released as open source, allowing others to expand the dataset or to construct their own datasets for needs which may be difficult to anticipate today. In total, the results of this project will enable any researcher (not just those at heavily-resourced institutions) to build AI systems which leverage large-scale synthetic indoor training data.The planned dataset construction pipeline will construct 3D scenes based on 2D floor plan datasets, which already exist at large scale. Using a machine-learning-based system previously developed by the investigators, these 2D floor plans will be converted to 3D models of empty houses. Then, each room in the house will be populated with objects in a plausible arrangement. Initially, this step will be performed by crowd workers on a platform such as Amazon Mechanical Turk. The workers will be instructed to place objects so as to match a photograph, where the photograph is chosen such that its (estimated) room geometry matches the geometry of the empty room to be populated. In a later stage of the project, rooms populated in this manner will be used to train a machine learning model which can automatically place objects based on an input photograph, thus further accelerating the dataset construction process.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

为了充当有用的家庭助理，机器人需要了解它们所看到的内容以及如何在室内环境中导航。目前解决这些问题的最先进方法依赖于机器学习，特别是深度学习，这需要深度学习。大量的标记数据（例如，许多带有每个像素标签的图像指示该像素处存在什么类型的对象），而不是要求人们费力地标记从现实世界空间捕获的数据，一种有前途的替代方法是使用*合成。 *3D场景：填充这些虚拟空间的 3D 对象可以配备诸如其对象类型之类的信息，这使得基本上可以“免费”创建大量标记的训练数据。 *社区标准的大型合成 3D 场景数据集 虽然存在一些合成 3D 场景数据集，但它们要么太小，要么由于 3D 版权问题而受到严格的使用限制（甚至诉讼）。模型，通常来自营利性公司。该项目将利用免费提供的 3D 内容构建一个大型数据集。该项目的主要贡献不仅仅是这个数据集，而且还是用于创建此类 3D 场景数据集的“可扩展管道”。该管道将​​作为开源发布，允许其他人扩展数据集或构建自己的数据集以满足今天可能难以预测的需求。总的来说，该项目的结果将使任何研究人员（而不仅仅是那些研究人员）受益。资源机构）构建人工智能系统利用大规模合成室内训练数据。计划的数据集构建管道将基于 2D 平面图数据集构建 3D 场景，这些数据集使用研究人员之前开发的基于机器学习的系统，这些 2D 平面图数据集已经大规模存在。计划将被转换为空房子的 3D 模型，然后，房子中的每个房间都将以合理的方式填充物体，最初，这一步将由亚马逊 Mechanical Turk 等平台上的工作人员执行。被指示将物体放置为匹配照片，选择照片使其（估计的）房间几何形状与要填充的空房间的几何形状相匹配。在项目的后期阶段，以这种方式填充的房间将用于训练机器学习。该模型可以根据输入照片自动放置物体，从而进一步加速数据集构建过程。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。