EAGER: Social Networks Based Concept Learning in Images

EAGER：基于社交网络的图像概念学习

• 批准号: 1552454
• 负责人: Bir Bhanu
• 金额: $ 20万
• 依托单位: University of California-Riverside
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2015
• 资助国家: 美国
• 起止时间: 2015-09-01 至 2018-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1552454&HistoricalAwards=false
• 关键词: EAGER; Social; Networks; Based; Concept

The need for easy, quick, and intuitive search of visual data at the conceptual level is universal. This project will explore a novel network analysis-based approach to searching visual data, ultimately leading to visual search engines that would make searching images as easy as searching by keywords is today. Accomplishing this requires new approaches to machine learning of visual concepts. This project proposes a formal framework that unifies the ideas from social networks and semantic concept learning so multiple semantic concepts can be learned with high confidence. Specifically, the approach utilizes hierarchical co-occurrence correlation among concepts as cues to help the detection of individual visual concepts. The sources for robustness are the learning of co-occurrence patterns, similar to community structures in a social network, and their refinement over time. The success of concept co-occurrence detection will simplify management of personal image data with automatic tagging. With semantically organized personal content, the preferences of a user can be learned to provide personalization of various contents that he/she consumes online. This will have a broad impact for diverse applications ranging from information technology to physical, life and social sciences to intelligence organizations to news bureaus. Forensics analysis of digital data would be greatly speeded up as humans do not have to sift through large amount of data. Extension of the techniques to video, music, and multi-modal data would also provide similar ease for content consumption. The research team provides an environment integrating education and workforce development with research, and with recruiting and retaining a diverse group of students. Complementing the research activities will be new initiatives in education and public outreach. The project develops a transformative approach to explore the acquisition and refinement of semantic visual concepts systematically. First, it discovers the hierarchical co-occurrence patterns of concepts as underlying community structures in the co-occurrence network. The co-occurrence patterns play roles similar to underlying scene concepts at a higher level of semantics. Second, it proposes an approach for selecting visually-consistent-semantic concepts. Since concepts vary in their visual complexity, visual-semantic relatedness of each concept is investigated by quantitatively measuring the within-concept visual variability and the visual distances to the other concepts such that they can be modeled more reliably and detected more easily. Third, the project introduces a novel image content descriptor called concept signature that can record both the semantic concept and the corresponding confidence value inferred from low-level features. Finally, the project proposes techniques for scalability to handle and evaluate the performance on large databases by developing open source techniques and software tools. The results will be broadly disseminated through the project website (http://vislab.ucr.edu/RESEARCH/VisualSemanticConcepts/VSC.php), via regular releases of software tools and offering tutorials/workshops at major IEEE/ACM conferences.

在概念层面上对视觉数据的简单，快速和直观的搜索的需求是普遍的。该项目将探索一种基于网络分析的新型方法来搜索视觉数据，最终导致视觉搜索引擎，这将使搜索图像与按关键字进行搜索一样容易。完成此操作需要新的方法来进行视觉概念的机器学习。该项目提出了一个正式的框架，该框架将社交网络和语义概念学习的想法统一，因此可以高度自信地学习多个语义概念。具体而言，该方法利用概念之间的层次结构同时相关性作为有助于检测单个视觉概念的提示。鲁棒性的来源是学习共发生模式，类似于社交网络中的社区结构以及它们随着时间的流逝而进行的完善。概念共同发生检测的成功将通过自动标记简化个人图像数据的管理。借助语义上有条理的个人内容，可以学会用户的偏好，以提供他/她在线消费的各种内容的个性化。这将对从信息技术到物理，生活和社会科学再到情报组织再到新闻局的各种应用产生广泛的影响。由于人类不必筛选大量数据，因此对数字数据的取证分析将大大加快。将技术扩展到视频，音乐和多模式数据也将提供类似的内容消耗。研究团队提供了一个环境，将教育和劳动力发展与研究融合在一起，并招募和保留各种各样的学生。补充研究活动将是教育和公共推广方面的新举措。该项目开发了一种变革性的方法来系统地探索语义视觉概念的获取和完善。首先，它发现概念的层次结构共汇模式，作为同时存在网络中的基本社区结构。共发生模式在更高层次的语义上扮演着类似于基础场景概念的角色。其次，它提出了一种选择视觉上一致的语义概念的方法。由于概念的视觉复杂性各不相同，因此通过定量测量概念内的视觉变异性和与其他概念的视觉距离来研究每个概念的视觉语义相关性，从而可以更可靠地对其进行建模和更容易检测到它们。第三，该项目介绍了一个名为Concept Signature的新型图像内容描述符，该描述可以记录从低级特征推断出的语义概念和相应的置信值。最后，该项目提出了通过开发开源技术和软件工具来处理和评估大数据库中性能的可伸缩性技术。结果将通过项目网站（http://vislab.ucr.edu/research/visualsemanticconcepts/vsc.php）广泛传播，通过定期发布软件工具的版本，并在Major IEEE/ACM会议上提供教程/工作室。