Modeling rich inter-image relationships in big visual collections

在大型视觉集合中建模丰富的图像间关系

基本信息

批准号：
1514512
负责人：
Alexei Efros
金额：
$ 22.59万
依托单位：
University of California-Berkeley
依托单位国家：
美国
项目类别：
Continuing Grant
财政年份：
2015
资助国家：
美国
起止时间：
2015-09-01 至 2017-08-31
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1514512&HistoricalAwards=false
关键词：
Modeling rich inter image relationships

项目摘要

The Directorate of Social, Behavioral and Economic Sciences offers postdoctoral research fellowships to provide opportunities for recent doctoral graduates to obtain additional training, to gain research experience under the sponsorship of established scientists, and to broaden their scientific horizons beyond their undergraduate and graduate training. Postdoctoral fellowships are further designed to assist new scientists to direct their research efforts across traditional disciplinary lines and to avail themselves of unique research resources, sites, and facilities, including at foreign locations. This postdoctoral fellowship supports a rising scientist in the interdisciplinary area overlapping computer vision and psychology, with a research project that investigates the web of relationships within visual data in both humans and machines. To a human observer, no photograph is an island: it is connected to the rest of the visual world by a web of similarities, associations, and other relationships. For example, two photos of Paris share a certain similarity; images of boats are associated with images of water; a photo of a tadpole and a photo of a frog show the same organism at two stages of life. In each of these cases, a human can readily reason about the link between two images. Not only can people identify that a relationship exists, but can also identify the nature of this relationship. These relationships shed light on how the human brain organizes visual information, and also give insight into how to build intelligent systems that automatically make visual connections. The latter will bring the field closer to producing an intelligent visual web, able to organize visual information in the same way as the current Internet is able to organize text. Computer vision scientists and psychologists have both studied relationships between visual data, but from different directions. In computer vision, the focus has been on models of natural image similarity. These models handle complex stimuli but are usually limited to one simple kind of relationship, namely similarity in appearance. Psychologists have studied a richer set of relationships - association, causation, analogy, antonymy, transformation, etc. - but their models usually only apply to simple, artificial stimuli. This project unites the best of both fields by modeling subtle visual relationships between complex, natural images. The objective is to model both which images humans consider to be related and how are those images related. An additional objective is to study how certain relationships, such as visual associations, can arise in an unsupervised manner from natural visual experience. This will help explain how humans might learn about the relationships in the first place. Better models of inter-image relationships will have deep implications across cognitive psychology. In particular, similarity and association play fundamental roles in theories of human learning and memory. A sense of similarity underlies our ability to learn from one visual experience and then apply our knowledge in a future, similar setting. The associations made from the experience additionally impact human memory of it. The present project also has applications toward computer vision systems. Reverse image search has recently become a popular tool. However, current systems are only able to retrieve look-alike images. If the computer is instead able to retrieve images linked by more diverse kinds of relationships, many possibilities open up. For example, one could imagine a system that lets users navigate through artistic styles, or that recommends shoes that match a pair of pants. If successful, this project could pave the way toward a world-wide web of visual connections that parallels the current web of hypertext connections.

社会，行为和经济科学局提供了博士后研究奖学金，为最近的博士毕业生提供了获得额外培训的机会，以在既定科学家的赞助下获得研究经验，并将其科学视野扩大到本科生和研究生培训之外。博士后奖学金的进一步旨在帮助新科学家在传统的纪律线上指导他们的研究工作，并利用独特的研究资源，站点和设施，包括在外国地点。这项博士后奖学金支持跨学科领域的一位不断上升的科学家与计算机视觉和心理学重叠，研究项目研究了人类和机器中视觉数据中的关系网络。对于人类的观察者来说，没有照片是一个岛屿：它通过相似之处，协会和其他关系的网络连接到视觉世界的其余部分。例如，巴黎的两张照片具有一定的相似性。船的图像与水的图像有关； t的照片和青蛙的照片在生活的两个阶段显示同一生物体。在每种情况下，人类都可以轻松地理解两个图像之间的联系。人们不仅可以识别存在关系，还可以识别这种关系的性质。这些关系阐明了人脑如何组织视觉信息，并洞悉如何构建自动建立视觉连接的智能系统。后者将使该领域更接近生产智能的视觉网络，能够以与当前Internet能够组织文本相同的方式来组织视觉信息。计算机视觉科学家和心理学家都研究了视觉数据之间的关系，但来自不同的方向。在计算机视觉中，重点是自然图像相似性的模型。这些模型处理复杂的刺激，但通常仅限于一种简单的关系，即外观上的相似性。心理学家研究了一组更丰富的关系 - 关联，因果关系，类比，反义词，转型等 - 但是它们的模型通常仅适用于简单的人造刺激。该项目通过对复杂的自然图像之间的微妙的视觉关系进行建模，将两个领域的最好。目的是建模人类认为是相关的图像以及这些图像如何相关的图像。另一个目标是研究某些关系（例如视觉关联）如何从自然的视觉体验中以无监督的方式出现。这将有助于解释人类首先如何了解这种关系。更好的模型间关系模型将在认知心理学上具有深刻的影响。特别是，相似性和关联在人类学习和记忆理论中扮演着基本角色。一种相似之处是我们从一种视觉体验中学习的能力，然后在未来的类似环境中运用我们的知识。从经验中建立的协会还影响了人类的记忆。本项目还针对计算机视觉系统有应用。反向图像搜索最近已成为一种流行的工具。但是，当前系统只能检索类似外观的图像。如果计算机能够检索通过更多种关系链接的图像，那么许多可能性就会开放。例如，人们可以想象一个系统，可以使用户可以通过艺术风格导航，或者推荐与一条裤子相匹配的鞋子。如果成功的话，该项目可能会为世界范围内的视觉连接网络铺平道路，该网络与当前的超文本连接网络平行。