EAGER: Exploring Multimedia Information Networks

EAGER：探索多媒体信息网络

• 批准号: 1144111
• 负责人: Thomas Huang
• 金额: $ 19.94万
• 依托单位: University of Illinois at Urbana-Champaign
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2011
• 资助国家: 美国
• 起止时间: 2011-08-01 至 2013-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1144111&HistoricalAwards=false
• 关键词: EAGER; Exploring; Multimedia; Information; Networks

This project is built on the interdisciplinary collaboration between multimedia and natural language processing (NLP) researchers. The goal of this exploratory research is to provide effective methods for organizing, searching, mining and reasoning with web-scale multimedia. The approach is based on formulation of a structured multimedia database, called Multimedia Information Networks (MINets) which enables new searching and mining paradigms such as keyword query of unlabeled image/video databases, query expansion in content-based image retrieval (CBIR), and measuring similarity between different modalities such as an image and a piece of text). In addition, MINets framework is expected to provide the ability to perform robust inference (e.g., recognizing objects and activities in images or videos) in the presence of noise and uncertainty. In its simplest form, a MINet is a graph where nodes are either concepts (text) or data (such as images), and links are ontological/semantic relationships between concepts, attachment of images to concepts, and visual similarity measures between images. Construction of an experimental MINets framework involves crawling the Web for a particular domain, gathering images with associated text and exploiting natural language processing, computer vision and mining techniques in establishing the concepts, associated images, interconnecting links and an ontology that supports inference. Validation against current web search engines and CBIR techniques is expected to provide a proof-of-concept for the novel MINets framework.This interdisciplinary exploratory project is expected to yield general theoretical and algorithmic MINets framework that will provide new searching, mining and reasoning capabilities for multimedia data. It will help to define new research areas in effective utilization of multimedia information sources for cross-media and cross-conceptual knowledge discovery and analysis, large-scale annotation, information fusion and inference. Project results, including open source software, annotated corpora, scoring metrics will be disseminated via project Website (https://netfiles.uiuc.edu/qi4/www/MINets.htm). This project will provide research opportunities for graduate and undergraduate students.

该项目建立在多媒体与自然语言处理（NLP）研究人员之间的跨学科合作之上。这项探索性研究的目的是提供有效的方法，用于通过网络规模的多媒体组织，搜索，采矿和推理。该方法是基于结构化多媒体数据库的公式，称为多媒体信息网络（嵌度），该数据库可实现新的搜索和采矿范例，例如未标记的图像/视频数据库的关键字查询，基于内容的图像retrieval（CBIR）中查询的扩展（CBIR），并测量不同模态之间的相似性，例如图像和图像和图像片段）。此外，有望在存在噪声和不确定性的情况下执行强大推断（例如，识别图像或视频中的对象和活动）的矿体框架。以最简单的形式，一个模丝是一个图形，节点是概念（文本）或数据（例如图像），链接是概念之间的本体论/语义关系，图像与概念的附件以及图像之间的视觉相似性测量值。 实验千分之一的框架的构建涉及爬网上的特定领域，收集具有相关文本的图像，并利用自然语言处理，计算机视觉和采矿技术，以建立概念，相关图像，相互联系的链接以及支持推理的本体论。对当前的Web搜索引擎和CBIR技术的验证有望为新型Minets框架提供概念概念。有望产生一般的理论和算法矿物框架，可为多媒体数据提供新的搜索，采矿和推理能力。它将有助于定义新的研究领域，以有效利用多媒体信息源用于跨媒体和跨概念知识发现和分析，大规模注释，信息融合和推理。项目结果，包括开源软件，带注释的Corpora，评分指标将通过项目网站（https://netfiles.uiuc.edu/qi4/www/minets.htm）传播。该项目将为研究生和本科生提供研究机会。