ITR: (NHS) - (dmc+sim): Computational Audio-Visual Scene Analysis (CAVSA)

ITR：(NHS) - (dmc sim)：计算视听场景分析 (CAVSA)

• 批准号: 0426627
• 负责人: Thomas Huang
• 金额: $ 60万
• 依托单位: University of Illinois at Urbana-Champaign
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2004
• 资助国家: 美国
• 起止时间: 2004-09-15 至 2008-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0426627&HistoricalAwards=false
• 关键词: ITR; NHS; dmc+sim; Computational; Audio

Abstract 0426627Huang, ThomasU of IllAudiovisual Scene AnalysisIn many applications such as Homeland Security, video surveillance is a crucial component. An important task is to extract moving objects (vehicles, people) and to separate the individual sounds (car noise, speech). But this is done usually separately, the former from the image sequences alone, and the latter from the audio (of the sound mixture). However, the two are closely coupled. We aim to do the two jointly, and term the task: Audiovisual Scene Analysis. The tool we shall use is Probabilistic Graphical Models, esp. Generative Models. A generative model includes hidden variables which affect the observed data (in this case, video with the associated audio).Using the Expectation-Maximization algorithms, these parameters can be estimated. The challenge is to come up with appropriate models such that the parameter estimation will lead to audiovisual object separation. Once the audiovisual objects in a video are separated, then techniques such as face recognition and speech analysis could be used to detect objects and events of importance to specific applications.

摘要0426627HUANG，iLlaudiovisual场景分析的托马苏（Thomasu）在许多应用程序中，例如国土安全性，视频监视是一个至关重要的组成部分。 一个重要的任务是提取移动的物体（车辆，人）并分离单个声音（汽车噪音，语音）。 但这通常是单独完成的，前者仅与图像序列，而后者来自音频（声音混合物）。 但是，两者是紧密耦合的。 我们的目标是共同执行这两个任务：视听场景分析。 我们将使用的工具是概率图形模型，特别是。生成模型。 生成模型包括影响观察到的数据的隐藏变量（在这种情况下，带有相关音频的视频）。使用期望最大化算法，可以估算这些参数。 挑战是提出适当的模型，以便参数估计将导致视听对象分离。 一旦视频中的视听对象分开，就可以使用诸如面部识别和语音分析之类的技术来检测对特定应用的对象和重要事件。