CAREER: Deadline-Ordered Multicast: A Network Primitive for Cloud-Native Distributed Systems

职业：按截止日期排序的多播：云原生分布式系统的网络原语

• 批准号: 2340748
• 负责人: Anirudh Sivaraman Kaushalram
• 金额: $ 80.12万
• 依托单位: New York University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2024
• 资助国家: 美国
• 起止时间: 2024-09-01 至 2029-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2340748&HistoricalAwards=false
• 关键词: CAREER; Deadline; Ordered; Multicast; Network

Computer systems spread out over multiple machines---also known as distributed systems---power most of today's digital world and the vast majority of web applications. Such digital systems include storage systems and search engine as well as web services. Spreading out digital systems across machines has multiple benefits including (1) the ability to survive machine failures by moving computation to a different machine and (2) the ability to leverage the capabilities of multiple machines to finish computations faster. To realize these benefits, the machines within a distributed system must coordinate amongst themselves to ensure the result is identical to an idealised illusion of executing the computation on a single machine. This project will make coordination significantly faster by developing a new abstraction, deadline-ordered multicast (DOM), which combines: (1) Multicast: The ability to simultaneously transmit a message from a sender to multiple receivers and (2) Synchronized clocks: Technology to ensure that, at any instant, clocks on various machines display the same value of time and that these clocks progress in lock step with each other. The project will demonstrate how DOM can be used to accelerate widely used computer systems, such as distributed databases, coordination services, and blockchains. The project will also train both undergraduate and graduate students using a variety of approaches to understand the benefits of synchronized clocks when thinking about distributed systems. Distributed systems are at the heart of computing today and include widely used systems such as distributed databases, fault-tolerant key-value stores, and distributed ledgers. At the core of these systems are distributed protocols such as crash-fault-tolerant consensus, concurrency control, and Byzantine-fault-tolerant consensus. Much effort has been expended on improving the performance of distributed protocols over the years including recent efforts that leverage richer network services to improve distributed protocols. Such services include switch multicast, programmable switches, programmable network-interface cards, in-network priority queues, and control over routing. However, many distributed systems today are deployed by cloud tenants, who have no access to such rich network services, making it impossible for them to benefit from the improved protocol performance enabled by such rich network services.This project will develop a new network primitive called deadline-ordered multicast (DOM) that will make it easier to construct high-performance distributed protocols on the public cloud. DOM leverages 2 key techniques: (1) the recent availability of synchronized clocks as a service, which now permit tightly synchronized clocks in the public cloud and (2) multicast, which struggled to find traction in the Internet, but is ideal for the one-sender-multi-receiver communication at the heart of distributed systems. DOM delivers a multicast message from a sender to multiple receivers at or after a message's deadline and delivers multiple messages in deadline order. DOM provides a consistent order (the order of deadlines) in which different receivers process a set of messages, accelerating several distributed protocols in the process. This proposal will develop the DOM abstraction, design an optimized DOM service, and prototype several applications demonstrating DOM's value.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

计算机系统分布在多台机器上---也称为分布式系统--- - 当今大多数数字世界和绝大多数Web应用程序的功能。 这样的数字系统包括存储系统和搜索引擎以及Web服务。将数字系统跨机器传播有多个好处，包括（1）通过将计算转移到其他机器以及（2）利用多台计算机功能以更快地完成计算的能力来生存的能力。为了实现这些好处，分布式系统中的机器必须彼此协调，以确保结果与在单个机器上执行计算的理想化幻觉相同。该项目将通过开发一个新的抽象，截止日期的多播（DOM）来大大更快，该项目结合在一起：（1）多播：能够从发送者传递消息到多个接收器的能力，并（2）同步时钟：技术以确保在各种机器上的时钟在各种时钟上的时钟和锁定时间的相同时钟，并在各个机器上都在锁定时锁定。 该项目将展示如何使用DOM加速使用广泛使用的计算机系统，例如分布式数据库，协调服务和区块链。该项目还将使用各种方法来培训本科生和研究生，以了解分布式系统时同步时钟的好处。分布式系统是当今计算的核心，包括广泛使用的系统，例如分布式数据库，容易耐故障的键值商店和分布式分类帐。这些系统的核心是分布式协议，例如易碰撞耐受性共识，并发控制和拜占庭式耐受耐受性共识。多年来，已经花费了很多努力来改善分布式协议的性能，包括最近的努力，以利用更丰富的网络服务来改善分布式协议。这样的服务包括开关多播，可编程开关，可编程网络接口卡，网络内优先级排队以及对路由的控制。但是，当今许多分布式系统由无法访问如此丰富的网络服务的云租户部署，这使得他们不可能从这样的丰富网络服务启用的改进的协议性能中受益。该项目将开发一个称为“截止日期订购的多播（DOM）”的新网络，这将使易于构建公共云上的高绩效协议更容易。 DOM利用2种关键技术：（1）最近同步时钟作为服务的最新可用性，现在允许在公共云中紧密同步时钟和（2）多播（2）多播，这些时钟很难在Internet中找到牵引力，但非常适合分布式系统的核心通信。 DOM在消息的截止日期或之后向多个接收器发送多播消息，并按照截止顺序提供多个消息。 DOM提供了一个一致的顺序（截止日期顺序），其中不同的接收器处理一组消息，从而加速了该过程中的几个分布式协议。该提案将开发DOM抽象，设计优化的DOM服务，并原型几种应用于DOM的价值的应用程序。该奖项反映了NSF的法定任务，并使用基金会的知识分子优点和更广泛的影响审查标准，认为值得通过评估来获得支持。