RI: Small: A Unified Compositional Model for Explainable Video-based Human Activity Parsing

RI：小型：用于可解释的基于视频的人类活动解析的统一组合模型

• 批准号: 1815561
• 负责人: Ying Wu
• 金额: $ 44.9万
• 依托单位: Northwestern University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-09-01 至 2023-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1815561&HistoricalAwards=false
• 关键词: RI; Small; Unified; Compositional; Model

An ultimate goal of computer vision is understanding scene and activities from images and video. This task involves many perceptual and cognitive processes at various semantic levels. A next step beyond visual classification is visual interpretation, that is, to explain the relations among visual entities through visual inference and reasoning. Due to the enormous variability across instances of this problem, semantic parsing for explaining a visual scene and activities is highly challenging. This project studies how the structural composition of visual entities can be used to overcome the diversity in the visual scene and activities. It advances and enrich the basic research of computer vision, and brings significant impact on many merging applications, including autonomous or assisted driving, intelligent robots, and intelligent video surveillance. This research also contributes to education through curriculum development, student training, and knowledge dissemination. It includes interactions with K-12 students for participation and research opportunities. This research is to develop a unified visual compositional model that can effectively learn complex semantic concepts in a scalable end-to-end fashion, while achieving good generalizability and providing explainable parsing of the visual data. The project is focused on: (1) a principled model and its theoretical foundation, by designing a stochastic grammar based on the probabilistic And/Or-Graph to model the structural composition; (2) an effective computational approach for learning and parsing, by exploiting data-driven pattern mining to discover structural components and by exploring how the patterns may be self-formed; (3) a solid case study on video human activity parsing and interpretation, by inferring the complex compositions of human actions, body movements, and interaction with the environment; and (4) tools and prototype systems for human articulated body pose estimation, contextual object discovery, and video-based human activity analysis and interpretation.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

计算机视觉的最终目标是了解图像和视频的场景和活动。这项任务涉及各种语义级别的许多感知和认知过程。视觉分类之外的下一步是视觉解释，即通过视觉推断和推理来解释视觉实体之间的关系。由于这个问题的实例之间存在巨大变化，用于解释视觉场景和活动的语义解析是极具挑战性的。该项目研究了如何使用视觉实体的结构组成来克服视觉场景和活动中的多样性。它进步并丰富了计算机视觉的基础研究，并对许多合并应用程序（包括自主或辅助驾驶，智能机器人和智能视频监视）产生重大影响。这项研究还通过课程发展，学生培训和知识传播来促进教育。它包括与K-12学生进行参与和研究机会的互动。 这项研究是开发一个统一的视觉组成模型，该模型可以以可扩展的端到端方式有效地学习复杂的语义概念，同时实现良好的概括性并提供可解释的视觉数据解析。该项目的重点是：（1）通过基于概率和/或图形来建模结构组成的随机语法，通过设计一个随机语法，是一种原则模型及其理论基础； （2）通过利用数据驱动的模式挖掘来发现结构成分并探索模式如何自我形成，一种有效的学习和解析计算方法； （3）通过推断人类行为，身体运动和与环境相互作用的复杂组成，对人类活动解析和解释的扎实案例研究； （4）用于人类表达的身体姿势估计，上下文对象发现以及基于视频的人类活动分析和解释的工具和原型系统。该奖项反映了NSF的法定任务，并被认为是通过基金会的智力优点和更广泛的影响来通过评估来支持的。

项目成果

Unsupervised Depth Completion and Denoising for RGB-D Sensors

RGB-D 传感器的无监督深度补全和去噪

• DOI: 10.1109/icra46639.2022.9812392
• 发表时间: 2022
• 期刊: 2022 International Conference on Robotics and Automation (ICRA
• 作者: Fan, Lei;Li, Yunxuan;Jiang, Chen;Wu, Ying
• 通讯作者: Wu, Ying

Uncertainty-Aware Score Distribution Learning for Action Quality Assessment

• DOI: 10.1109/cvpr42600.2020.00986
• 发表时间: 2020-01-01
• 期刊: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Proceedings
• 作者: Yansong Tang;Zanlin Ni;Jie Zhou
• 通讯作者: Jie Zhou

Deeply Learned Compositional Models for Human Pose Estimation

• DOI: 10.1007/978-3-030-01219-9_12
• 发表时间: 2018-01-01
• 期刊: COMPUTER VISION - ECCV 2018, PT III
• 作者: Tang, Wei;Yu, Pei;Wu, Ying
• 通讯作者: Wu, Ying

Order-Preserving Wasserstein Discriminant Analysis

• DOI: 10.1109/iccv.2019.00998
• 发表时间: 2019-10
• 期刊: 2019 IEEE/CVF International Conference on Computer Vision (ICCV)
• 作者: Bing Su;Jiahuan Zhou;Ying Wu

A Modulation Module for Multi-task Learning with Applications in Image Retrieval

• DOI: 10.1007/978-3-030-01246-5_25
• 发表时间: 2018-07
• 作者: Xiangyu Zhao;Haoxiang Li;Xiaohui Shen;Xiaodan Liang;Ying Wu