CI-P: Planning for AudioNet: A New Community Infrastructure for Audio Annotations for Acoustic Event Identification

CI-P：规划 AudioNet：用于声学事件识别的音频注释的新社区基础设施

• 批准号: 1629990
• 负责人: Gerald Friedland
• 金额: $ 10万
• 依托单位: International Computer Science Institute
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2016
• 资助国家: 美国
• 起止时间: 2016-07-01 至 2018-12-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1629990&HistoricalAwards=false
• 关键词: CI; Planning; AudioNet; New; Community

This effort lays the groundwork for AudioNet, a public-domain corpus of audio labels for the nearly 800,000 videos in the open-access YFCC100M dataset. Audio information provides an important complement to visual information in the automatic analysis of video data, allowing systems to detect situations that may not be clearly identifiable from the visual stream alone. However, there are as yet no truly large-scale labeled audio datasets of the kind needed as input to build flexible, accurate analysis systems. Creating such a large-scale corpus will serve as an impetus for better multimedia algorithms to be developed by more researchers and computer science students, translating into an impact on the everyday life of the public at large. Social media videos are increasingly used for scientific research, as they provide an opportunity to observe and model many phenomena in the social sciences, economics, meteorology, and medicine. New capabilities for content analysis will therefore impact many scientific fields. In addition, audio analysis could be used in real-time security surveillance and in robotics applications like autonomous vehicles and household robots to aid and monitor the elderly.AudioNet is part of a multi-institution collaboration, the Multimedia Commons initiative, which is developing a variety of resources around the YFCC100M dataset of Creative Commons-licensed photos and videos. AudioNet is annotating the audio tracks from the YFCC100M videos, focusing on audio concepts. Audio concepts can be thought of as acoustic "objects": concrete, localizable units of sound like "crowd cheering" or "fire alarm". The approach will be modeled on ImageNet, an image dataset labeled and organized using the WordNet hierarchy of synsets (groups of synonyms); ImageNet has enabled major enabled advances in image processing. However, while ImageNet focuses largely on entities (noun synsets), audio data is inherently temporal. The label set for AudioNet will therefore focus on events and actions, though similarly organized using semantic resources like WordNet.

这项工作为 AudioNet 奠定了基础，AudioNet 是一个公共域音频标签语料库，包含开放访问 YFCC100M 数据集中的近 800,000 个视频。音频信息在视频数据的自动分析中为视觉信息提供了重要的补充，使系统能够检测仅从视觉流中无法清楚识别的情况。然而，目前还没有真正大规模的标记音频数据集，可以作为构建灵活、准确的分析系统的输入。创建如此大规模的语料库将推动更多研究人员和计算机科学专业的学生开发更好的多媒体算法，从而对广大公众的日常生活产生影响。社交媒体视频越来越多地用于科学研究，因为它们提供了观察和建模社会科学、经济学、气象学和医学中许多现象的机会。因此，内容分析的新功能将影响许多科学领域。此外，音频分析还可用于实时安全监控以及自动驾驶汽车和家用机器人等机器人应用，以帮助和监控老年人。AudioNet 是多机构合作多媒体共享计划的一部分，该计划正在开发一个围绕知识共享许可的照片和视频的 YFCC100M 数据集的各种资源。 AudioNet 正在注释 YFCC100M 视频中的音轨，重点关注音频概念。音频概念可以被认为是声学“对象”：具体的、可本地化的声音单元，如“人群欢呼”或“火警”。该方法将以 ImageNet 为模型，ImageNet 是一个使用 WordNet 同义词集层次结构进行标记和组织的图像数据集； ImageNet 使图像处理领域取得了重大进展。然而，虽然 ImageNet 主要关注实体（名词同义词集），但音频数据本质上是时间性的。因此，AudioNet 的标签集将重点关注事件和操作，尽管使用 WordNet 等语义资源进行类似的组织。