BIGDATA: F: Collaborative Research: From Visual Data to Visual Understanding

BIGDATA:F:协作研究:从视觉数据到视觉理解

基本信息

  • 批准号:
    1633310
  • 负责人:
  • 金额:
    $ 40万
  • 依托单位:
  • 依托单位国家:
    美国
  • 项目类别:
    Standard Grant
  • 财政年份:
    2016
  • 资助国家:
    美国
  • 起止时间:
    2016-09-01 至 2020-08-31
  • 项目状态:
    已结题

项目摘要

The field of visual recognition, which focuses on creating computer algorithms for automatically understanding photographs and videos, has made tremendous gains in the past few years. Algorithms can now recognize and localize thousands of objects with reasonable accuracy as well as identify other visual content, such as scenes and activities. For instance, there are now smart phone apps that can automatically sift through a user's photos and find all party pictures, or all pictures of cars, or all sunset photos. However, the type of "visual understanding" done by these methods is still rather superficial, exhibiting mostly rote memorization rather than true reasoning. For example, current algorithms have a hard time telling if an image is typical (e.g., car on a road) or unusual (e.g., car in the sky), or answering simple questions about a photograph, e.g., "what are the people looking at?", "what just happened?", "what might happen next?" A central problem is that current methods lack the data about the world outside of the photograph. To achieve true human-like visual understanding, computers will have to reason about the broader spatial, temporal, perceptual, and social context suggested by a given visual input. This project is using big visual data to gather large-scale deep semantic knowledge about how events, physical and social interactions, and how people perceive the world and each other. The research focuses on developing methods to capture and represent this knowledge in a way that makes it broadly applicable to a range of visual understanding tasks. This will enable novel computer algorithms that have a deeper, more human-like, understanding of the visual world and can effectively function in complex, real-world situations and environments. For example, if a robot can predict what a person might do next in a given situation, then the robot can better aid the person in their task. Broader impacts will include new publicly-available software tools and data that can be used for various visual reasoning tasks. Additionally, the project will have a multi-pronged educational component, including incorporating aspects of the research in the graduate teaching curriculum, undergraduate and K-12 outreach, as well as special mentoring and focused events for advancement of women in computer science.The main technical focus of this project is to advance computational recognition efforts toward producing a general human-like visual understanding of images and video that can function on previously unseen data, unseen tasks and settings. The aim of this project is to develop a new large-scale knowledge base called the visual Memex that extracts and stores vast set of visual relationships between data items in a multi-graph representation, with nodes corresponding to data items and edges indicating different types of relationships. This large knowledge base will be used in a lambda-calculus-powered reasoning engine to make inferences about visual data on a global scale. Additionally, the project will test computational recognition algorithms on several visual understanding tasks designed to evaluate progress on a variety of aspects of visual understanding, including: linguistic (evaluating our understanding about imagery through language tasks such as visual question-answering), to purely visual (evaluating our understanding of spatial context through visual fill-in-the-blanks), to temporal (evaluating our temporal understanding by predicting future states), to physical (evaluating our understanding of human-object and human-scene interactions by predicting affordances). Datasets, knowledge base, and evaluation tools will be hosted on the project web site (http://www.tamaraberg.com/grants/bigdata.html).
视觉识别领域的重点是创建用于自动理解照片和视频的计算机算法,在过去的几年中取得了巨大的收益。算法现在可以以合理的精度识别和本地化数千个对象,并识别其他视觉内容,例如场景和活动。例如,现在有一些智能手机应用程序可以自动筛选用户的照片,并找到所有派对图片,或所有汽车的图片或所有日落照片。但是,这些方法所做的“视觉理解”类型仍然相当肤浅,主要是死记硬背的记忆,而不是真实的推理。例如,当前的算法很难告诉图像是典型的(例如,道路上的汽车)还是不寻常的(例如,天空中的汽车),或回答有关照片的简单问题,例如,“人们在看什么?”,“看什么?”,“刚刚发生了什么?”一个核心问题是,当前的方法缺乏照片之外的世界数据。为了获得真正的类似人类的视觉理解,计算机将不得不推理给给定的视觉输入提出的更广泛的空间,时间,感知和社会环境。该项目正在使用大型视觉数据来收集有关事件,身体和社交互动以及人们如何看待世界和彼此的大规模深层语义知识。该研究重点是开发以一种使其广泛适用于一系列视觉理解任务的方式来捕获和表示这些知识的方法。这将使新型计算机算法具有更深,更人性化的对视觉世界的理解,并且可以在复杂的,现实世界中的情况和环境中有效发挥作用。例如,如果机器人可以预测一个人在给定情况下接下来会做什么,那么机器人可以更好地帮助该人执行其任务。更广泛的影响将包括可用于各种视觉推理任务的新的公共可用软件工具和数据。此外,该项目将具有多种要求的教育组成部分,包括将研究的各个方面纳入研究生教学课程,本科和K-112宣传,以及在计算机科学领域促进女性的特殊指导和重点事件。该项目的主要技术重点是该项目的主要技术识别能够促进像人类型的视觉识别和视频效果,以前能够进行视觉识别和以前的视频效果。该项目的目的是开发一个名为“ Visual Memex”的新的大规模知识库,该知识库提取和存储数据项之间的大量视觉关系,其中的节点对应于数据项和边缘指示不同类型的关系。这个庞大的知识库将用于由兰巴达 - 钙的推理引擎中,以在全球范围内推断出视觉数据。此外,该项目将在几种视觉理解任务上测试计算识别算法,旨在评估视觉理解的各个方面的进步,包括:语言(评估我们通过语言任务的理解(评估我们的意象),例如视觉问题,纯粹的视觉效率(通过填写填充的情况来评估我们对空间的理解,对我们的临时理解(评估我们的视觉上下文)(评估我们的时间)(我们对时间的理解)(评估我们的时间)(我们对时间的理解)(对时间)进行理解(计时)(blanks firce)(计算我们的时间),通过预测负担能力)。数据集,知识库和评估工具将托管在项目网站(http://www.tamaraberg.com/grants/bigdata.html)上。

项目成果

期刊论文数量(12)
专著数量(0)
科研奖励数量(0)
会议论文数量(0)
专利数量(0)
Discovering Visual Patterns in Art Collections with Spatially-consistent Feature Learning
通过空间一致的特征学习发现艺术收藏中的视觉模式
RANSAC-Flow: generic two-stage image alignment
  • DOI:
    10.1007/978-3-030-58548-8_36
  • 发表时间:
    2020-04
  • 期刊:
  • 影响因子:
    0
  • 作者:
    XI Shen;Franccois Darmon;Alexei A. Efros;Mathieu Aubry
  • 通讯作者:
    XI Shen;Franccois Darmon;Alexei A. Efros;Mathieu Aubry
Space-Time Correspondence as a Contrastive Random Walk
  • DOI:
  • 发表时间:
    2020-06
  • 期刊:
  • 影响因子:
    0
  • 作者:
    A. Jabri;Andrew Owens;Alexei A. Efros
  • 通讯作者:
    A. Jabri;Andrew Owens;Alexei A. Efros
Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks
Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Prediction
{{ item.title }}
{{ item.translation_title }}
  • DOI:
    {{ item.doi }}
  • 发表时间:
    {{ item.publish_year }}
  • 期刊:
  • 影响因子:
    {{ item.factor }}
  • 作者:
    {{ item.authors }}
  • 通讯作者:
    {{ item.author }}

数据更新时间:{{ journalArticles.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ monograph.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ sciAawards.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ conferencePapers.updateTime }}

{{ item.title }}
  • 作者:
    {{ item.author }}

数据更新时间:{{ patent.updateTime }}

Alexei Efros其他文献

Alexei Efros的其他文献

{{ item.title }}
{{ item.translation_title }}
  • DOI:
    {{ item.doi }}
  • 发表时间:
    {{ item.publish_year }}
  • 期刊:
  • 影响因子:
    {{ item.factor }}
  • 作者:
    {{ item.authors }}
  • 通讯作者:
    {{ item.author }}

{{ truncateString('Alexei Efros', 18)}}的其他基金

Modeling rich inter-image relationships in big visual collections
在大型视觉集合中建模丰富的图像间关系
  • 批准号:
    1514512
  • 财政年份:
    2015
  • 资助金额:
    $ 40万
  • 项目类别:
    Continuing Grant
Data-Driven Appearance Transfer for Realistic Image Synthesis
用于真实图像合成的数据驱动的外观传输
  • 批准号:
    0541230
  • 财政年份:
    2006
  • 资助金额:
    $ 40万
  • 项目类别:
    Continuing Grant
CAREER: Geometrically Coherent Image Interpretation
职业:几何相干图像解释
  • 批准号:
    0546547
  • 财政年份:
    2006
  • 资助金额:
    $ 40万
  • 项目类别:
    Continuing Grant
NIRT: Nanoscale Metalic Photonic Crystals; Fabrication, Physical Properties, and Applications
NIRT:纳米级金属光子晶体;
  • 批准号:
    0102964
  • 财政年份:
    2001
  • 资助金额:
    $ 40万
  • 项目类别:
    Continuing Grant
Study of Inhomogeneous State of Two-Dimensional Electron Quantum Liquid
二维电子量子液体非均匀态研究
  • 批准号:
    9116748
  • 财政年份:
    1992
  • 资助金额:
    $ 40万
  • 项目类别:
    Continuing Grant

相似国自然基金

数智背景下的团队人力资本层级结构类型、团队协作过程与团队效能结果之间关系的研究
  • 批准号:
    72372084
  • 批准年份:
    2023
  • 资助金额:
    40 万元
  • 项目类别:
    面上项目
在线医疗团队协作模式与绩效提升策略研究
  • 批准号:
    72371111
  • 批准年份:
    2023
  • 资助金额:
    41 万元
  • 项目类别:
    面上项目
面向人机接触式协同作业的协作机器人交互控制方法研究
  • 批准号:
    62373044
  • 批准年份:
    2023
  • 资助金额:
    50 万元
  • 项目类别:
    面上项目
基于数字孪生的颅颌面人机协作智能手术机器人关键技术研究
  • 批准号:
    82372548
  • 批准年份:
    2023
  • 资助金额:
    49 万元
  • 项目类别:
    面上项目
A-型结晶抗性淀粉调控肠道细菌协作产丁酸机制研究
  • 批准号:
    32302064
  • 批准年份:
    2023
  • 资助金额:
    30 万元
  • 项目类别:
    青年科学基金项目

相似海外基金

BIGDATA: IA: Collaborative Research: Asynchronous Distributed Machine Learning Framework for Multi-Site Collaborative Brain Big Data Mining
BIGDATA:IA:协作研究:用于多站点协作大脑大数据挖掘的异步分布式机器学习框架
  • 批准号:
    2348159
  • 财政年份:
    2023
  • 资助金额:
    $ 40万
  • 项目类别:
    Standard Grant
BIGDATA: IA: Collaborative Research: Intelligent Solutions for Navigating Big Data from the Arctic and Antarctic
BIGDATA:IA:协作研究:导航北极和南极大数据的智能解决方案
  • 批准号:
    2308649
  • 财政年份:
    2022
  • 资助金额:
    $ 40万
  • 项目类别:
    Standard Grant
BigData:IA:Collaborative Research: TIMES: A tensor factorization platform for spatio-temporal data
BigData:IA:协作研究:TIMES:时空数据张量分解平台
  • 批准号:
    2034479
  • 财政年份:
    2020
  • 资助金额:
    $ 40万
  • 项目类别:
    Standard Grant
BIGDATA: Collaborative Research: F: Holistic Optimization of Data-Driven Applications
BIGDATA:协作研究:F:数据驱动应用程序的整体优化
  • 批准号:
    2027516
  • 财政年份:
    2020
  • 资助金额:
    $ 40万
  • 项目类别:
    Standard Grant
BIGDATA: F: Collaborative Research: Practical Analysis of Large-Scale Data with Lyme Disease Case Study
BIGDATA:F:协作研究:莱姆病案例研究大规模数据的实际分析
  • 批准号:
    1934319
  • 财政年份:
    2019
  • 资助金额:
    $ 40万
  • 项目类别:
    Standard Grant
{{ showInfoDetail.title }}

作者:{{ showInfoDetail.author }}

知道了