Efficient and reliable coded distributed computing

高效可靠的编码分布式计算

• 批准号: 570977-2021
• 负责人: Ardakani, Masoud
• 金额: $ 3.64万
• 依托单位: University of Alberta
• 依托单位国家: 加拿大
• 项目类别: Alliance Grants
• 财政年份: 2021
• 资助国家: 加拿大
• 起止时间: 2021-01-01 至 2022-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=722762
• 关键词: Efficient; reliable; coded; distributed; computing

Many modern ICT applications work with data at scale and demand massive computations that cannot be performed in a single computer. This has led to the wide use of distributed computing, where a massive computational task is distributed among a large number of computing nodes in a communication network. In real-life, some of these computing nodes fail to deliver their task due to software/hardware failures, handling other tasks for the network, leaving the network, etc. These straggling nodes (typically around 5% of the processing nodes) result in unpredictable network performance and can significantly prolong job completion. Currently, redundancy in the form of repeating the tasks is implemented to combat the stragglers.Error-correcting codes offer an opportunity to combat stragglers at a much lower cost, reduced communication load, higher success rate, and with added security/privacy benefits. They also create the opportunity of using a large number of very low-cost hardware by the network to reliably finish a massive job in a short time. In this project(i) we will design various low-complexity error-correction coding algorithms that are feasible for large-scale distributed computing, hence enabling the network to handle data at scale reliably;(ii) we will design task scheduling algorithms that optimally distribute and schedule the tasks in the network in order to minimize the completion time/cost, with guaranteed success.We anticipate this project to significantly improve cloud services by developing coded distributed computation and task allocation/scheduling algorithms that (i) reduce the completion time and communication costs, (ii) very efficiently use the available resources, (iii) have low implementation complexity, and (iv) provide added privacy/security. In addition, our algorithms can be used in real-life applications such as telepresence, telehealth, augmented reality, distributed database management systems, real-time process control and more.

许多现代的ICT应用程序与大规模数据合作，并要求在一台计算机中执行大量计算。这导致了分布式计算的广泛使用，其中大量计算任务分布在通信网络中的大量计算节点中。在现实生活中，由于软件/硬件故障，处理网络的其他任务，离开网络等，这些计算节点中的一些无法实现其任务。这些散布的节点（通常约为处理节点的5％左右）导致无法预测的网络性能，并且可以显着延长工作完成。当前，重复任务的形式的冗余已被实施以对抗散乱者。纠正的代码为以低得多的成本，降低的沟通负载，更高的成功率以及增加的安全/隐私收益来打击Stragglers提供了一个机会。他们还创造了机会，可以通过网络使用大量非常低成本的硬件来在短时间内可靠完成大量工作。 In this project(i) we will design various low-complexity error-correction coding algorithms that are feasible for large-scale distributed computing, hence enabling the network to handle data at scale reliably;(ii) we will design task scheduling algorithms that optimally distribute and schedule the tasks in the network in order to minimize the completion time/cost, with guaranteed success.We anticipate this project to significantly improve cloud services by developing编码的分布式计算和任务分配/调度算法（i）降低完成时间和通信成本，（ii）非常有效地使用可用资源，（iii）具有较低的实现复杂性，并且（iv）提供了添加的隐私/安全性。此外，我们的算法可用于现实生活中的应用，例如远程兴趣，远程医疗，增强现实，分布式数据库管理系统，实时过程控制等。