CSR: Medium: Very Large Scale Consistent DHTs

CSR：中：超大规模一致的 DHT

• 批准号: 0963754
• 负责人: Thomas Anderson
• 金额: $ 119.92万
• 依托单位: University of Washington
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2010
• 资助国家: 美国
• 起止时间: 2010-03-15 至 2016-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0963754&HistoricalAwards=false
• 关键词: CSR; Medium; Very; Large; Scale; CSR; Medium; Very; Large; Scale

Modern distributed systems increasingly rely on distributed storage and lookup services. Existing storage and lookup solutions, however, provide only a subset of high availability, high scalability, and strong consistency semantics. In this project, we develop Harmony, a distributed hash table (DHT) aimed at providing very large scale applications with highly scalable, consistent and available storage and lookup. We address many significant challenges in delivering on this goal. First, the Internet has unpredictable node and communication failures, and preserving consistency in this context is a difficult task. To address this issue, we provide the abstraction of a collection of self-managing groups that coordinate to ensure atomic updates to distributed state. Second, consistency often comes at the cost of reduced availability. In our system, consistency is an inviolable safety property; availability is provided through replication. Third, coordination mechanisms for consistent replication and atomic updates often result in performance penalties. A key insight in our work is that one can improve performance with coordination mechanisms for delegation and autonomous execution. Finally, by isolating most of the communication necessary to preserve consistency within a group, we both simplify our implementation and improve its scalability. Further, adaptation to changing workloads and resource constraints is easier because it can take place within a framework for the consistent update of distributed state. If successful, the resulting storage abstraction should greatly simplify the development of complex distributed applications.

现代分布式系统越来越依赖分布式存储和查找服务。 但是，现有的存储和查找解决方案仅提供高可用性，高可扩展性和强烈一致性语义的子集。在这个项目中，我们开发了Harmony，这是一个分布式哈希表（DHT），旨在为非常大的应用程序提供高度可扩展，一致和可用的存储和查找。我们解决了实现这一目标的许多重大挑战。首先，互联网具有不可预测的节点和通信失败，并且在这种情况下保持一致性是一项艰巨的任务。 为了解决这个问题，我们提供了一组自我管理组的抽象，以确保分布式状态的原子更新。 其次，一致性通常以减少可用性为代价。 在我们的系统中，一致性是一种不可侵犯的安全性。可用性是通过复制提供的。 第三，用于一致复制和原子更新的协调机制通常会导致绩效惩罚。 我们工作中的一个关键见解是，可以通过授权和自主执行的协调机制来提高性能。 最后，通过隔离保持组中一致性所需的大多数沟通，我们既简化了我们的实施又提高其可扩展性。 此外，适应更改工作负载和资源约束更容易，因为它可以在分布式状态的一致更新框架内进行。如果成功，则最终的存储抽象应大大简化复杂分布式应用程序的开发。