ITR: A User-centric Content-based Approach to Indexing, Query and Retrieval of Music through Signal Processing and Knowledge-based Methods

ITR：一种以用户为中心、基于内容的方法，通过信号处理和基于知识的方法对音乐进行索引、查询和检索

• 批准号: 0219912
• 负责人: Shrikanth Narayanan
• 金额: $ 50万
• 依托单位: University of Southern California
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2002
• 资助国家: 美国
• 起止时间: 2002-08-15 至 2006-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0219912&HistoricalAwards=false
• 关键词: ITR; User; centric; Content; based

This research project will develop methods for content-based indexing of music databases, using a combination of signal processing and knowledge-based methods, design of statistical algorithms for enabling queries using sung or hummed melodies, and design of robust search techniques for retrieving the queried information, especially in the presence of uncertainty. The research approach, which is based on statistical modeling, is user-centric and comprises three major components: (1) Representation and Indexing - musical information utilizing theories and knowledge about human musical intuition, and music perception and cognition. (2) Query Formulation and Interaction Modality - algorithm design for enabling interaction with music data through humming, a natural activity. (3) Search and Retrieval - algorithm design to match user query against the database that will be robust to uncertainties and errors in the query generation.The architecture will include a front-end recognizer, that converts the humming signal to notes using a statistical pattern recognition approach, which interfaces with a back-end music database that is indexed using perceptually viable features. The search process matching a query against indices is formulated as a statistical information retrieval problem. The project will employ progressively rich indexing representation including repeating patterns for recurring themes, chord, beat, and key information. Derivation of such music-theoretic knowledge will benefit from a principled approach of mathematically modeling tonality in music. The statistical framework allows for handling variability and uncertainty in query formulation and retrieval. It also enables providing for quality of solution in the query results.This work will contribute not only to the specific economically significant application area of music, but also to general information science knowledge on how to index and search humanly-meaningful patterns in complex sensory data. The project will also serve as a vehicle to foster cross-disciplinary graduate and undergraduate research and education.

该研究项目将开发基于内容的音乐数据库索引方法，结合使用信号处理和基于知识的方法，设计统计算法以支持使用演唱或哼唱的旋律进行查询，以及设计用于检索查询的强大搜索技术信息，尤其是在存在不确定性的情况下。 该研究方法基于统计模型，以用户为中心，包括三个主要组成部分：（1）表示和索引——利用有关人类音乐直觉、音乐感知和认知的理论和知识的音乐信息。 (2) 查询制定和交互模式 - 通过哼唱（一种自然活动）与音乐数据交互的算法设计。 (3) 搜索和检索 - 将用户查询与数据库相匹配的算法设计，该算法对查询生成中的不确定性和错误具有鲁棒性。该架构将包括一个前端识别器，它使用统计模式将嗡嗡声信号转换为音符识别方法，它与后端音乐数据库交互，该数据库使用感知可行的特征进行索引。 将查询与索引相匹配的搜索过程被表述为统计信息检索问题。 该项目将采用逐渐丰富的索引表示，包括重复主题、和弦、节拍和关键信息的重复模式。 这种音乐理论知识的推导将受益于对音乐音调进行数学建模的原理方法。 统计框架允许处理查询制定和检索中的可变性和不确定性。 它还能够在查询结果中提供高质量的解决方案。这项工作不仅有助于音乐的特定经济应用领域，而且有助于了解如何在复杂的感官数据中索引和搜索对人类有意义的模式的一般信息科学知识。 该项目还将作为促进跨学科研究生和本科生研究和教育的工具。