CRII: CSR: Large-scale Systems Software Atop Scale-out In-memory Storage

CRII：CSR：横向扩展内存存储之上的大型系统软件

• 批准号: 1566175
• 负责人: Ryan Stutsman
• 金额: $ 17.49万
• 依托单位: University of Utah
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2016
• 资助国家: 美国
• 起止时间: 2016-05-15 至 2019-04-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1566175&HistoricalAwards=false
• 关键词: CRII; CSR; Large; scale; Systems

Research over the past 5 years on in-memory data center storage has resulted in systems that can provide on-demand access to billions of pieces of information per second. However, the benefits of these fast systems has not resulted in more powerful and interesting applications for users. The key problem is that these new storage systems are stripped down to provide speed and can only fulfill very basic requests for information. To compensate, applications must make many requests to these storage systems to perform the same operations they were able to perform with a single request to older database systems. The result is that applications spend most of the performance benefits they gain from these systems overcoming their limited programming model.This work seeks to understand the hidden costs of the limited interfaces of these large-scale in-memory key-value stores and to alleviate those costs with a new stored procedure model uniquely suited to these systems. To explore these costs, data-intensive applications from three different domains will be prototyped on today?s low-latency key-value stores, first with their conventional interfaces and then again with custom application logic built into the storage system. The results of this exploration will motivate and aid in the design of the main contribution of this work: a generalized model for safely collocating application logic inside of key-value stores. Application prototypes will include a scale-out database kernel, a simple scale-out graph database, and large-scale machine learning algorithms.A rich programming model for developing large-scale software systems will lower the barrier for building information-intensive applications: an art that requires specialized expertise. Low-latency systems equipped with fast stored procedures will be a powerful tool for building new, data-intensive systems in industry, science, and the military. They will enable deep real-time analysis of social and natural graphs; rich, interactive worlds where millions of users manipulate and modify a shared environment; the fine-grained coordination and routing of millions of autonomous vehicles on the highways in a metropolitan area; and real-time decision support for military intelligence. All components will be built as a practical and usable software system with all development done publicly and available as open-source.

过去 5 年对内存数据中心存储的研究已经产生了每秒可以按需访问数十亿条信息的系统。然而，这些快速系统的好处并没有为用户带来更强大、更有趣的应用程序。关键问题是这些新的存储系统为了提供速度而进行了精简，并且只能满足非常基本的信息请求。作为补偿，应用程序必须向这些存储系统发出许多请求，以执行与向旧数据库系统发出单个请求时能够执行的操作相同的操作。 结果是，应用程序花费了从这些系统克服其有限的编程模型所获得的大部分性能优势。这项工作旨在了解这些大规模内存中键值存储的有限接口的隐藏成本，并减轻这些成本专门适合这些系统的新存储过程模型的成本。为了探索这些成本，来自三个不同领域的数据密集型应用程序将在当今的低延迟键值存储上进行原型设计，首先使用其传统接口，然后再次使用存储系统中内置的自定义应用程序逻辑。这项探索的结果将激励并帮助设计这项工作的主要贡献：用于在键值存储内部安全地配置应用程序逻辑的通用模型。应用程序原型将包括横向扩展数据库内核、简单横向扩展图数据库和大规模机器学习算法。用于开发大规模软件系统的丰富编程模型将降低构建信息密集型应用程序的门槛：需要专门知识的艺术。配备快速存储过程的低延迟系统将成为在工业、科学和军事领域构建新的数据密集型系统的强大工具。它们将能够对社交和自然图进行深入的实时分析；丰富的交互式世界，数百万用户操纵和修改共享环境；对大都市区高速公路上数百万辆自动驾驶车辆进行细粒度的协调和路线安排；以及军事情报的实时决策支持。所有组件都将构建为实用且可用的软件系统，所有开发均公开完成并作为开源提供。

项目成果

Tailwind: Fast and Atomic RDMA-based Replication

Tailwind：基于 RDMA 的快速原子复制

• DOI: 10.1016/j.ijdrr.2021.102057
• 发表时间: 2018-07-11
• 期刊: International journal of disaster risk reduction
• 作者: Y. Taleb;Ryan Stutsman;Gabriel Antoniu;Toni Cortes
• 通讯作者: Toni Cortes

Splinter: Bare-Metal Extensions for Multi-Tenant Low-Latency Storage

Splinter：用于多租户低延迟存储的裸机扩展

• DOI: 10.14778/3229863.3229868
• 发表时间: 2018-10-08
• 期刊: Proc. VLDB Endow.
• 作者: C. Kulkarni;S. Moore;Mazhar Naqvi;Tian Zhang;R. Ricci;Ryan Stutsman
• 通讯作者: Ryan Stutsman