NRI: Real-Time Semantic Computer Vision for Co-Robotics

NRI：协作机器人的实时语义计算机视觉

基本信息

批准号：
1637941
负责人：
Nuno Vasconcelos
金额：
$ 71.91万
依托单位：
University of California-San Diego
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2016
资助国家：
美国
起止时间：
2016-09-01 至 2020-08-31
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1637941&HistoricalAwards=false
关键词：
NRI Real Time Semantic Computer

项目摘要

This project develops real-time object recognition algorithms that generate extensive semantic object descriptions as a side effect of recognition. This side-information includes perceived object costs, attributes (object properties), and affordances (actions afforded by objects). With these, the act of recognizing a "door knob" would automatically produce the information that this is a "flexible" object, "made of metal," which "can be grasped" and "can be twisted," but "cannot be eaten." For robotics, this information is sometimes more important than the recognition of the object itself. The project enables robots to perform zero shot learning, e.g. learn to recognize door knobs by simply being told that these are objects that "are flexible, made of metal, can be grasped and twisted but not eaten." The research has applicability in areas such as manufacturing, intelligent systems, assisted living, and homeland security. Educationally, the project provides an exciting opportunity for undergraduate research.This research develops new methods for top-down (task-driven) regularization of deep learning algorithms, though a combination of structural and loss-based regularizers. Structural regularizers constrain object and scene recognition models to guarantee speed and automatic generation of rich mid-level semantic (MLS) descriptions as a side effect of recognition. Loss-based regularizers penalize errors in the multiple semantic outputs of these models, enabling simultaneously high performance in object recognition, MLS predictions, and zero-shot learning. The resulting learning algorithms will endow robots with human-like abilities to infer rich MLS descriptions of objects and scenes as a "side effect" of object recognition and scene classification, in real-time. These contributions will be developed in the context of a new co-robotics problem, person-following unmanned aerial vehicles, where computer vision plays a mission critical role for tasks such as control and semantic motion planning but whose requirements in terms of speed and MLS inference are far superior to what is feasible today.

该项目开发了实时对象识别算法，该算法生成广泛的语义对象描述作为识别的副作用。此侧信息包括感知的对象成本，属性（对象属性）和负担（对象提供的操作）。有了这些，识别“门旋钮”的行为会自动产生这是“柔性”对象，由金属制成的物体，“可以抓住”和“可以扭曲”，但“不能食用”。对于机器人技术，这些信息有时比对象本身的识别更重要。该项目使机器人可以执行零射击学习，例如学会通过简单地被告知这些物体“柔韧，由金属制成，可以被握住，但不会吃掉”来识别门旋钮。”该研究在制造，智能系统，辅助生活和国土安全方面具有适用性。从教育上讲，该项目为本科研究提供了一个令人兴奋的机会。这项研究开发了新的（任务驱动）深度学习算法正规化的新方法，尽管结合了结构性和基于损失的正规化器。结构正规化器将对象和场景识别模型限制，以确保速度和自动生成丰富的中级语义（MLS）描述作为识别的副作用。基于损耗的正规化器会在这些模型的多个语义输出中惩罚错误，从而在对象识别，MLS预测和零局学习中同时具有高性能。由此产生的学习算法将赋予人类般的能力，以实时推断对象和场景的丰富MLS描述作为对象识别和场景分类的“副作用”。这些贡献将在一个新的共同机器人问题的背景下开发，该问题是以人为目标的无人驾驶汽车，在这些问题上，计算机视觉在控制和语义运动计划等任务中起着至关重要的作用，但其在速度和MLS推理方面的要求远远超过了当今可行的。

项目成果

期刊论文数量（17）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

Rethinking Differentiable Search for Mixed-Precision Neural Networks

DOI：
10.1109/cvpr42600.2020.00242
发表时间：
2020-01-01
期刊：
2020 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR)
影响因子：
0
作者：
Cai, Zhaowei;Vasconcelos, Nuno
通讯作者：
Vasconcelos, Nuno

Object based Scene Representations using Fisher Scores of Local Subspace Projections

DOI：
发表时间：
2016-12
期刊：
影响因子：
0
作者：
Mandar Dixit;N. Vasconcelos
通讯作者：
Mandar Dixit;N. Vasconcelos

RESOUND: Towards Action Recognition Without Representation Bias

DOI：
10.1007/978-3-030-01231-1_32
发表时间：
2018-01-01
期刊：
COMPUTER VISION - ECCV 2018, PT VI
影响因子：
0
作者：
Li, Yingwei;Li, Yi;Vasconcelos, Nuno
通讯作者：
Vasconcelos, Nuno

Deep Scene Image Classification with the MFAFVNet

DOI：
10.1109/iccv.2017.613
发表时间：
2017-10
期刊：
2017 IEEE International Conference on Computer Vision (ICCV)
影响因子：
0
作者：
Yunsheng Li;Mandar Dixit;N. Vasconcelos
通讯作者：
Yunsheng Li;Mandar Dixit;N. Vasconcelos

Bidirectional Learning for Domain Adaptation of Semantic Segmentation

DOI：
10.1109/cvpr.2019.00710
发表时间：
2019-01-01
期刊：
2019 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2019)
影响因子：
0
作者：
Li, Yunsheng;Yuan, Lu;Vasconcelos, Nuno
通讯作者：
Vasconcelos, Nuno