III: Large: Collaborative Research: SciDB - An Array Oriented Data Management System for Massive Scale Scientific Data

III：大型：协作研究：SciDB - 用于大规模科学数据的面向数组的数据管理系统

• 批准号: 1110948
• 负责人: Jignesh Patel
• 金额: $ 37.07万
• 依托单位: University of Wisconsin-Madison
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2011
• 资助国家: 美国
• 起止时间: 2011-09-01 至 2016-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1110948&HistoricalAwards=false
• 关键词: III; Large; Collaborative; Research; SciDB

This collaborative project brings together expertise of five research teams at Brown University (IIS-1111423), University of Washington (IIS-1110370), Massachusetts Institute of Technology (IIS-1111371), Portland State University (IIS-1110917) and University of Wisconsin-Madison (IIS-1111423). Scientific data management has traditionally been performed using the file system, at best using files structured according to a low-level data format. Higher-level data management infrastructure has been task-specific and not reusable in different domains, resulting in millions of dollars of duplicated implementation effort by scientists to manage their data. The goal of this project is the development of a scientific database (SciDB), a system designed and optimized for scientific applications. The aim of SciDB is to do for science what relational databases did for the business world, namely to provide a high performance, commercial-quality and scalable data management system appropriate for many science domains.In contrast to existing database systems, SciDB is based on a multidimensional array data model and includes multiple features specific to science and critical for science: provenance, uncertainty, versions, time travel, science-specific operations, and in situ data processing. No existing system offers all these features in a single, highly scalable engine. SciDB thus significantly advances the state-of-the-art in data management in addition to supporting domain scientists in data-driven knowledge discovery. The intellectual merit of SciDB is in exploring novel, high performance solutions to nested array storage, parallel array query optimization and execution, array language design, and time travel.The primary broader impact of SciDB is on the community of scientists who benefit from the tool. By keeping scientists "in the loop" in the design of the system from the outset, the project delivers software that is broadly usable to the community. The proposal also funds participation in a series of workshops that seek to engage even more of the science community. SciDB is an open-source effort, with an initial prototype (http://www.scidb.org/) already downloaded by hundreds of users. Finally, the PIs have a strong track record of delivering robust data management software that is widely used and involving students in the process, including students from under-represented groups. Further information can be found on the project web page (http://database.cs.brown.edu/projects/scidb).

该合作项目汇集了布朗大学 (IIS-1111423)、华盛顿大学 (IIS-1110370)、麻省理工学院 (IIS-1111371)、波特兰州立大学 (IIS-1110917) 和威斯康星大学五个研究团队的专业知识-麦迪逊（IIS-1111423）。传统上，科学数据管理是使用文件系统执行的，最多使用根据低级数据格式构建的文件。 更高级别的数据管理基础设施是特定于任务的，并且不可在不同领域重复使用，导致科学家花费数百万美元的重复实施工作来管理他们的数据。 该项目的目标是开发科学数据库（SciDB），这是一个为科学应用而设计和优化的系统。 SciDB 的目标是像关系数据库为商业世界所做的那样为科学做事，即提供适合许多科学领域的高性能、商业质量和可扩展的数据管理系统。与现有的数据库系统相比，SciDB 基于多维数组数据模型，包括多个特定于科学和对科学至关重要的特征：来源、不确定性、版本、时间旅行、特定于科学的操作和原位数据处理。现有系统还没有在一个高度可扩展的引擎中提供所有这些功能。因此，SciDB 除了支持领域科学家进行数据驱动的知识发现之外，还显着推进了数据管理的最先进水平。 SciDB 的智力优势在于探索针对嵌套数组存储、并行数组查询优化和执行、数组语言设计和时间旅行的新型高性能解决方案。SciDB 的主要更广泛影响在于从该工具中受益的科学家社区。 通过从一开始就让科学家“参与”系统设计，该项目提供了可供社区广泛使用的软件。该提案还资助参加一系列研讨会，旨在吸引更多科学界人士参与。 SciDB 是一项开源项目，其初始原型 (http://www.scidb.org/) 已被数百名用户下载。最后，PI 在提供广泛使用的强大数据管理软件方面拥有良好的记录，并让学生（包括来自代表性不足群体的学生）参与这一过程。更多信息可以在项目网页 (http://database.cs.brown.edu/projects/scidb) 上找到。