EAGER: Towards Human Centered Visual Understanding: Exploring the Intended and Interpreted Meaning of Images in Social Multimedia

EAGER：迈向以人为本的视觉理解：探索社交多媒体中图像的预期和解释意义

• 批准号: 1350763
• 负责人: Hong Man
• 金额: $ 19.92万
• 依托单位: Stevens Institute of Technology
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2013
• 资助国家: 美国
• 起止时间: 2013-09-15 至 2018-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1350763&HistoricalAwards=false
• 关键词: EAGER; Towards; Human; Centered; Visual

This project explores a new direction in computer vision, which is to model the context dependent visual semantics associated with images in social multimedia. The context dependent visual semantics, e.g., the intended and perceived sentiment of an image in social multimedia, are dynamically formed based on the various contextual information associated with it. This is different from the static visual semantics that conventional computer vision research focused on studying, such as the object category presented in the image.The project develops a set of new networked and context aware probabilistic latent semantic models, which integrate situated contextual information into visual content analysis for modeling context dependent visual semantics. The research team is verifying two hypotheses: 1) the context dependent semantics needs to be holistically modeled and jointly inferred from a collection of related images; and 2) related context dependent visual semantics, such as intended and perceived meaning of an image, also needs to be jointly modeled for more robust recognition.The project is integrated with education through training graduate and undergraduate students. The outcome of the research can be applied to many domains, such as targeted online advertisements; open source information analysis and social event prediction; and social multimedia security.

该项目探索了计算机视觉的一个新方向，即对与社交多媒体中的图像相关的上下文依赖的视觉语义进行建模。依赖于上下文的视觉语义，例如社交多媒体中图像的预期和感知情感，是基于与其关联的各种上下文信息动态形成的。这与传统计算机视觉研究重点研究的静态视觉语义不同，例如图像中呈现的对象类别。该项目开发了一套新的网络化和上下文感知的概率潜在语义模型，将情境上下文信息集成到视觉中用于建模上下文相关视觉语义的内容分析。研究团队正在验证两个假设：1）上下文依赖语义需要整体建模并从相关图像集合中联合推断； 2）相关的上下文依赖视觉语义，例如图像的预期意义和感知意义，也需要联合建模以实现更强大的识别。该项目通过培训研究生和本科生与教育相结合。研究成果可以应用于许多领域，例如定向在线广告；开源信息分析和社会事件预测；和社交多媒体安全。