CSR: Medium: Declarative Programmable Storage

CSR：媒介：声明式可编程存储

• 批准号: 1764102
• 负责人: Peter Alvaro
• 金额: $ 85万
• 依托单位: University of California-Santa Cruz
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-08-15 至 2023-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1764102&HistoricalAwards=false
• 关键词: CSR; Medium; Declarative; Programmable; Storage

Everyone who processes large amounts of data - biologists and physicists, data scientists, and computer scientists of all stripes - must rely on distributed storage systems. Storage systems deal with conflicting demands: they must accommodate the changing workloads of the domain experts who use them, while ensuring good performance by taking advantage of changing hardware such as faster disks and networks - and guaranteeing that data is safely stored. This means that they must be rigid and flexible at the same time. Declarative programmable storage attempts to provide the best of both worlds by providing a simple, concise language for describing new storage interfaces, and using techniques from database optimization to automatically find correct and efficient implementations of the interface.Declarative programmable storage identifies key opportunities to apply classic data management systems concepts - such as declarative language and cost-based query optimization - to the domain of large-scale storage systems, allowing for cross-pollination between the storage, data management and distributed systems research communities. Additionally, this project requires novel research into cost modeling and query planning in the context of the storage domain. By decoupling the specification of new storage application programming interfaces (a task that should be undertaken by domain experts - e.g., physicists - in a high-level language) from their concrete implementations (a low-level task that can be automated), this project will free the users of distributed storage systems from the need to be experts in the internals of such systems. Consequently, these domain experts will be able to focus on innovation within their own area of expertise rather than becoming expert storage system programmers as well. These application programming interfaces can be specified once and re-optimized whenever device characteristics change. Hence the labor of these users will be made "future-proof" against the rapid evolution of storage hardware, software and configurations. Scientific results, software, and data from the work performed under this grant will be made available to the public under a free and open source software license. The project repository will be available at https://declstore.soe.ucsc.edu/ backed up by departmental backup systems to ensure long-term preservation of digital artifacts. In the long-term, this project will aim to engage open-source software communities to maintain and evolve software created under this grant.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

每个处理大量数据的人——生物学家、物理学家、数据科学家和各行各业的计算机科学家——都必须依赖分布式存储系统。 存储系统处理相互冲突的需求：它们必须适应使用它们的领域专家不断变化的工作负载，同时通过利用不断变化的硬件（例如更快的磁盘和网络）来确保良好的性能，并保证数据的安全存储。 这意味着它们必须同时具有刚性和灵活性。声明式可编程存储试图通过提供一种简单、简洁的语言来描述新的存储接口，并使用数据库优化技术来自动找到正确且高效的接口实现，从而实现两全​​其美。声明式可编程存储确定了应用经典的关键机会将数据管理系统概念（例如声明性语言和基于成本的查询优化）引入大规模存储系统领域，从而允许存储、数据管理和分布式系统研究社区之间的交叉授粉。 此外，该项目需要对存储域环境中的成本建模和查询规划进行新颖的研究。通过将新存储应用程序编程接口的规范（应由领域专家（例如物理学家）用高级语言执行的任务）与其具体实现（可以自动化的低级任务）分离，该项目将使分布式存储系统的用户不再需要成为此类系统内部的专家。 因此，这些领域专家将能够专注于自己专业领域内的创新，而不是成为专家存储系统程序员。 这些应用程序编程接口可以指定一次，并在设备特性发生变化时重新优化。 因此，这些用户的劳动将能够“适应未来”存储硬件、软件和配置的快速发展。 此项资助下所开展工作的科学成果、软件和数据将根据免费开源软件许可向公众开放。该项目存储库将在 https://declstore.soe.ucsc.edu/ 上提供，并由部门备份系统备份，以确保数字文物的长期保存。从长远来看，该项目的目标是吸引开源软件社区来维护和发展根据该赠款创建的软件。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力优势和更广泛的影响进行评估，被认为值得支持审查标准。

项目成果

Processing Particle Data Flows with SmartNICs

使用 SmartNIC 处理粒子数据流

• 发表时间: 2022-09
• 期刊: Proceedings of the 26th Annual 2022 IEEE High Performance Extreme Computing (IEEE-HPEC 2022
• 作者: Jianshen Liu; Carlos Maltzahn
• 通讯作者: Carlos Maltzahn

Scale-out Edge Storage Systems with Embedded Storage Nodes to Get Better Availability and Cost-Efficiency At the Same Time

具有嵌入式存储节点的横向扩展边缘存储系统，同时获得更好的可用性和成本效益

• 发表时间: 2020-01
• 期刊: HotEdge'20
• 作者: Jianshen Liu; Matthew Leon
• 通讯作者: Matthew Leon

Zero-Cost, Arrow-Enabled Data Interface for Apache Spark

适用于 Apache Spark 的零成本、支持 Arrow 的数据接口

• DOI: 10.1109/bigdata52589.2021.9671595
• 发表时间: 2021-06-24
• 期刊: 2021 IEEE International Conference on Big Data (Big Data)
• 作者: Sebastiaan Alvarez Rodriguez;Jayjeet Chakraborty;Aaron Chu;I. Jimenez;J. LeFevre;C. Maltzahn;Ale;ru Uta;ru
• 通讯作者: ru

SkyhookDM: Data Processing in Ceph with Programmable Storage

SkyhookDM：使用可编程存储在 Ceph 中进行数据处理

• 发表时间: 2020-01
• 期刊: USENIX login;
• 作者: Jeff LeFevre; Carlos Maltzahn
• 通讯作者: Carlos Maltzahn

Skyhook: Towards an Arrow-Native Storage System

Skyhook：迈向 Arrow 原生存储系统

• DOI: 10.48550/arxiv.2204.06074
• 发表时间: 2022-04-12
• 期刊: 2022 22nd IEEE International Symposium on Cluster, Cloud and Internet Computing (CCGrid)
• 作者: Jayjeet Chakraborty;I. Jimenez;Sebastiaan Alvarez Rodriguez;Ale;ru Uta;ru;J. LeFevre;C. Maltzahn
• 通讯作者: C. Maltzahn