CRII: NeTS: Scaling Distributed Storage with Programmable Switches

CRII：NeTS：使用可编程交换机扩展分布式存储

• 批准号: 1755646
• 负责人: Xin Jin
• 金额: $ 17.43万
• 依托单位: Johns Hopkins University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-04-01 至 2020-03-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1755646&HistoricalAwards=false
• 关键词: CRII; NeTS; Scaling; Distributed; Storage

Modern Internet services, such as search, social networking and e-commerce, depend critically on high performance distributed storage. As these services scale to billions of users, system operators are increasingly relying on in-memory storage to meet the necessary throughput and latency demands. A major challenge in scaling distributed storage is coping with skewed, dynamic workloads, which can lead to severe load imbalance and performance degradations. While traditional flash-based and disk-based storage can be balanced using a fast in-memory caching layer, server-based caching does not work for in-memory storage, because there is little difference in performance between the caching and storage layers. This project investigates a new distributed storage architecture that leverages the power and flexibility of new generation programmable switches to cache data in the network. The project goal is to not only improve the performance and reliability of cloud systems in practice, but also provide new architectural and theoretical insights on classical distributed systems topics. This project will foster the collaboration between the networking and systems communities in the new area of co-designing networks and systems with programmable switches.The project will develop a new architecture to provide high aggregate throughput and low latency even under highly-skewed and rapidly-changing workloads. There are two major technical thrusts. The first one is to design the switch cache with limited switch functionalities and resources. The approach will be to exploit the match-action tables, register arrays, and multi-pipeline, multi-stage structure of modern switch designs to efficiently index and pack variable-length objects into limited switch table and memory resources. The second research thrust is to design the overall system to achieve scalability, consistency and fault-tolerance. While these are classical distributed systems topics, they will be revisited in a new type of heterogeneous system which consists of fast switch caches and general-purpose storage servers. A prototype system will be constructed using both programmable software and hardware switches, and extensively evaluated with a wide range of realistic and synthetic workloads.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

搜索，社交网络和电子商务等现代互联网服务在很大程度上取决于高性能分布式存储。随着这些服务规模扩展到数十亿个用户，系统运营商越来越依靠内存存储来满足必要的吞吐量和延迟需求。扩展分布式存储的主要挑战是应对偏斜的动态工作负载，这可能导致严重的负载失衡和性能降解。虽然传统的基于闪存的基于磁盘和基于磁盘的存储可以使用快速的内存中心层平衡，但基于服务器的缓存对内存存储不起作用，因为缓存和存储层之间的性能几乎没有差异。该项目研究了一种新的分布式存储架构，该架构利用了新一代可编程开关的功能和灵活性到网络中的缓存数据。项目目标不仅是在实践中提高云系统的性能和可靠性，而且还为经典分布式系统主题提供了新的架构和理论见解。该项目将促进与可编程开关的共同设计网络和系统新领域的网络和系统社区之间的合作。该项目将开发一个新的体系结构，即使在高度稳定且迅速变化的工作负载下，也可以提供高汇总吞吐量和低潜伏期。有两个主要的技术推力。第一个是使用有限的开关功能和资源设计开关高速缓存。该方法将是利用匹配表，寄存器数组和多台线的多阶段结构的现代开关设计的多阶段结构，以有效地索引和包装可变长度对象到有限的开关表和内存资源中。第二项研究的目的是设计整体系统，以实现可伸缩性，一致性和容忍度。尽管这些是经典的分布式系统主题，但它们将通过一种新型的异质系统进行重新访问，该系统由快速开关的缓存和通用存储服务器组成。将使用可编程软件和硬件开关构建原型系统，并通过广泛的现实和综合工作负载进行广泛的评估。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的知识分子优点和更广泛的审查标准通过评估来进行评估的。

项目成果

Harmonia: Near-Linear Scalability for Replicated Storage with In-Network Conflict Detection

• DOI: 10.14778/3368289.3368301
• 发表时间: 2019-04
• 期刊: Proc. VLDB Endow.
• 作者: Hang Zhu;Zhihao Bai;Jialin Li;Ellis Michael;Dan R. K. Ports;I. Stoica;Xin Jin

ASAP: Fast, Approximate Graph Pattern Mining at Scale

• 发表时间: 2018
• 作者: A. Iyer;Zaoxing Liu;Xin Jin;S. Venkataraman;V. Braverman;I. Stoica

AWStream: adaptive wide-area streaming analytics

• DOI: 10.1145/3230543.3230554
• 发表时间: 2018-08
• 期刊: Proceedings of the 2018 Conference of the ACM Special Interest Group on Data Communication
• 作者: Ben Zhang;Xin Jin;Sylvia Ratnasamy;J. Wawrzynek;Edward A. Lee

Neural packet classification

• DOI: 10.1145/3341302.3342221
• 发表时间: 2019-02
• 期刊: Proceedings of the ACM Special Interest Group on Data Communication
• 作者: Eric Liang;Hang Zhu;Xin Jin;I. Stoica

DistCache: Provable Load Balancing for Large-Scale Storage Systems with Distributed Caching

• 发表时间: 2019-01
• 作者: Zaoxing Liu;Zhihao Bai;Zhenming Liu;Xiaozhou Li;Changhoon Kim;V. Braverman;Xin Jin;I. Stoica