CSR-PDOS-Content-Searchable Storage for Feature-Rich Data

CSR-PDOS-功能丰富的数据的内容可搜索存储

• 批准号: 0509447
• 负责人: Kai Li
• 金额: $ 90万
• 依托单位: Princeton University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2005
• 资助国家: 美国
• 起止时间: 2005-07-01 至 2009-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0509447&HistoricalAwards=false
• 关键词: CSR; PDOS; Content; Searchable; Storage

Storage capacity and data volume have been doubling every 18 months during the pasttwo decades. A key challenging issue in building next-generation storage systems is tomanage massive amounts of feature-rich (non-text) data, which has dominated theincreasing volume of digital information. Comparing noisy, feature-rich data requires fast similarity match instead of exact match, and thus exploring such data requires similaritysearch instead of exact search. Current file systems are designed for named text files;they do not have mechanisms to manage feature-rich data. To date, there is no practicalstorage system with the ability to do similarity search for noisy, high-dimensional dataand there is no index engine design for efficient similarity search. This researchaddresses this problem by studying how to design and implement a content-addressableand -searchable storage (CASS) system to manage and explore diverse feature-rich data.The system includes a built-in similarity search engine for general-purpose, noisy, highdimensionalmetadata using compact data structures and novel indexing methods. Theresearch will also develop segmentation methods and feature extraction methods foraudio, image and genomic data, and develop similarity search benchmarks and toevaluate the CASS system.This research will advance knowledge and understanding in the area of storage systemdesigns such as data structures, mechanisms, and APIs for managing, searching andexploring noisy, high-dimensional feature-rich data. The research will accelerate thedevelopment of next-generation storage systems which will revolutionize how to access,search, explore and manage massive amounts of feature-rich data in many disciplines.

在过去二十年中，存储容量和数据量每 18 个月就翻一番。构建下一代存储系统的一个关键挑战是管理大量功能丰富的（非文本）数据，这些数据主导了不断增长的数字信息量。比较嘈杂、特征丰富的数据需要快速相似性匹配而不是精确匹配，因此探索此类数据需要相似性搜索而不是精确搜索。当前的文件系统是为命名文本文件设计的；它们没有管理功能丰富的数据的机制。迄今为止，还没有实用的存储系统能够对嘈杂的高维数据进行相似性搜索，也没有用于高效相似性搜索的索引引擎设计。本研究通过研究如何设计和实现内容可寻址和可搜索存储（CASS）系统来管理和探索各种特征丰富的数据来解决这个问题。该系统包括一个内置的相似性搜索引擎，用于通用、嘈杂的高维元数据使用紧凑的数据结构和新颖的索引方法。研究还将开发音频、图像和基因组数据的分割方法和特征提取方法，并开发相似性搜索基准并评估 CASS 系统。这项研究将增进对数据结构、机制和 API 等存储系统设计领域的知识和理解用于管理、搜索和探索嘈杂、高维、特征丰富的数据。该研究将加速下一代存储系统的开发，这将彻底改变如何访问、搜索、探索和管理多个学科中大量、功能丰富的数据。