EAGER: Leveraging Synthetic Data for Visual Reasoning and Representation Learning with Minimal Human Supervision

EAGER：在最少的人类监督下利用合成数据进行视觉推理和表示学习

• 批准号: 1748387
• 负责人: Yong Jae Lee
• 金额: $ 20万
• 依托单位: University of California-Davis
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-08-15 至 2020-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1748387&HistoricalAwards=false
• 关键词: EAGER; Leveraging; Synthetic; Data; Visual

This project investigates how synthetic data created using computer graphics can be used for developing algorithms that understand visual data. Synthetic data provides flexibility that is difficult to obtain with real-world imagery, and enables opportunities to explore problems that would be difficult to solve with real-world imagery alone. This project develops new algorithms for reasoning about object occlusions, and for self-supervised representation learning, in which useful image features are developed without the aid of human-annotated semantic labels. The project provides new algorithms that have the potential to benefit applications in autonomous systems and security. In addition to scientific impact, the project performs complementary educational and outreach activities that engage students in research and STEM.This research explores novel algorithms that learn from synthetic data for visual reasoning and representation learning. While the use of synthetic data has a long history in computer vision, it has mainly been used to complement natural image data to solve standard tasks. In contrast, this project uses synthetic data to make advances in relatively unexplored problems, in which ground-truth is difficult to obtain given real-world imagery. The project consists of three major thrusts, each of which exploits the fact that a user has full control of everything that happens in a synthetic dataset. In Thrust I, it investigates a novel approach to representation learning using synthetic data, and in Thrust II, it extends the algorithm to disentangle task-specific and general-purpose features. Finally, in Thrust III, it explores a novel approach for reasoning about object occlusions.

该项目研究了如何使用计算机图形创建的合成数据来开发了解视觉数据的算法。合成数据提供了很难通过现实世界图像获得的灵活性，并使机会可以探索仅凭现实世界图像就难以解决的问题。该项目开发了用于推理对象闭塞的新算法，并为自我监督的表示学习而言，在没有人类通知的语义标签的情况下开发了有用的图像特征。该项目提供了新算法，有可能使自主系统和安全性中的应用程序受益。除了科学影响外，该项目还进行了互补的教育和外展活动，这些活动使学生参与研究和茎。这项研究探讨了从合成数据中学习的新型算法，以进行视觉推理和表示。虽然合成数据的使用在计算机视觉方面具有悠久的历史，但它主要用于补充自然图像数据以解决标准任务。相比之下，该项目使用合成数据来取得相对尚未探索的问题的进步，在该问题中很难获得地面真相。该项目由三个主要推力组成，每个推力都利用了用户对合成数据集中发生的一切完全控制的事实。在推力I中，它研究了一种使用合成数据的新型表示学习方法，并且在推力II中，它将算法扩展到解开特定于任务特定和通用特征的算法。最后，在推力III中，它探讨了一种用于推理对象闭塞的新方法。

项目成果

HPLFlowNet: Hierarchical Permutohedral Lattice FlowNet for Scene Flow Estimation on Large-Scale Point Clouds

• DOI: 10.1109/cvpr.2019.00337
• 发表时间: 2019-06
• 期刊: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
• 作者: Xiuye Gu;Yijie Wang;Chongruo Wu;Yong Jae Lee;Panqu Wang

SinGAN-GIF: Learning a Generative Video Model from a Single GIF

• DOI: 10.1109/wacv48630.2021.00135
• 发表时间: 2021-01
• 期刊: 2021 IEEE Winter Conference on Applications of Computer Vision (WACV)
• 作者: Rajat Arora;Yong Jae Lee

FineGAN: Unsupervised Hierarchical Disentanglement for Fine-Grained Object Generation and Discovery

• DOI: 10.1109/cvpr.2019.00665
• 发表时间: 2018-11
• 期刊: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
• 作者: Krishna Kumar Singh;Utkarsh Ojha;Yong Jae Lee

Elastic-InfoGAN: Unsupervised Disentangled Representation Learning in Class-Imbalanced Data

• 发表时间: 2020
• 作者: Utkarsh Ojha;Krishna Kumar Singh;Cho-Jui Hsieh;Yong Jae Lee

MixNMatch: Multifactor Disentanglement and Encoding for Conditional Image Generation

• DOI: 10.1109/cvpr42600.2020.00806
• 发表时间: 2019-11
• 期刊: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
• 作者: Yuheng Li;Krishna Kumar Singh;Utkarsh Ojha;Yong Jae Lee