Advanced coding solutions for large-scale data storage & communication

适用于大规模数据存储的高级编码解决方案

• 批准号: RGPIN-2017-04119
• 负责人: Ardakani, Masoud
• 金额: $ 2.4万
• 依托单位: University of Alberta
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2018
• 资助国家: 加拿大
• 起止时间: 2018-01-01 至 2019-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=644098
• 关键词: Advanced; coding; solutions; large; scale

Motivation***The global IP traffic is growing rapidly due to increased demand for online services such as cloud storage, video on demand, live streaming, and file sharing. These online services, which are responsible for the bulk of the Internet traffic, heavily rely on distributed storage systems (DSS). Hence, around the world, the number and size of DSSs have rapidly increased in recent years. In fact, DSSs are already among the top contributors of carbon emission in the modern world. In the US, for example, they are only second to the airline industry and expected to be the top contributor by 2020. Thus, research on efficient data storage techniques for DSSs has recently attracted a lot of interest both in academia and in industry.***Background***Hardware and software failures in a DSS result in data loss and unavailability. Naturally, many service providers keep multiple copies of data for reliability and availability. The downside, of course, is the huge storage overhead (typically 200%) and the associated maintenance costs. As a solution, recently, error correction coding is suggested for DSSs. Unfortunately, most existing error correction codes cannot be used in these large-scale applications, because the repair traffic that they create is beyond the network capacity of the DSSs. Hence, a strong need for more efficient codes, tailored for large-scale applications, is felt. ***Goals ***In this research program, using our past knowledge of modern coding and our recent activities on coding for DSSs, we will develop efficient and scalable error correction coding solutions for distributed storage systems. We will also seek theoretical results on the fundamental relations among different performance measures (such as coding overhead, code reliability, repair traffic, update complexity, etc.). ***There are a large number of high-impact open problems in this area all suitable for training HQPs.***Impact***This research proposal follows several goals that are exceedingly important to both practitioners and theorists in the area of coding and information theory. Improving the efficiency of large-scale data storage not only improves our online experience, but also significantly reduces the carbon footprint of DSSs. We hope our results throughout this work and our contribution with other Canadian researchers turn Canada into a major scientific center in this field, which can technically and scientifically support any data center located in Canada. In fact, since cooling is a major cost in data centers, Canada because of its colder climate and its safety is considered as an ideal place to host data centers. ***In the past decade, the world has seen a significant increase in online activity. As a result, large-scale data storage/communication currently is and will continue to be a big engineering challenge. Our activity in this field will train HQPs that will be in high demand and involved in very high impact engineering activities.

动机***由于云存储、视频点播、直播和文件共享等在线服务的需求增加，全球 IP 流量正在快速增长。这些负责大部分互联网流量的在线服务严重依赖分布式存储系统（DSS）。因此，近年来，全球决策支持系统的数量和规模迅速增加。事实上，DSS 已经成为现代世界碳排放的最大贡献者之一。例如，在美国，它们仅次于航空业，预计到 2020 年将成为最大的贡献者。因此，针对 DSS 的高效数据存储技术的研究最近引起了学术界和工业界的极大兴趣。* **背景*** DSS 中的硬件和软件故障会导致数据丢失和不可用。当然，为了可靠性和可用性，许多服务提供商会保留多个数据副本。当然，缺点是巨大的存储开销（通常为 200%）和相关的维护成本。作为一种解决方案，最近建议对 DSS 进行纠错编码。不幸的是，大多数现有的纠错码无法在这些大规模应用中使用，因为它们产生的修复流量超出了 DSS 的网络容量。因此，强烈需要为大规模应用量身定制的更高效的代码。 ***目标*** 在本研究计划中，利用我们过去对现代编码的了解以及最近在 DSS 编码方面的活动，我们将为分布式存储系统开发高效且可扩展的纠错编码解决方案。我们还将寻求不同性能指标（例如编码开销、代码可靠性、修复流量、更新复杂性等）之间基本关系的理论结果。 ***该领域有大量高影响力的开放问题，都适合培训 HQP。***影响力***本研究提案遵循的几个目标对于编码领域的实践者和理论家来说都非常重要和信息论。提高大规模数据存储的效率不仅可以改善我们的在线体验，还可以显着减少DSS的碳足迹。我们希望我们整个工作的成果以及我们与其他加拿大研究人员的贡献使加拿大成为该领域的主要科学中心，可以在技术和科学上支持位于加拿大的任何数据中心。事实上，由于冷却是数据中心的主要成本，加拿大因其寒冷的气候和安全性而被认为是托管数据中心的理想地点。 ***在过去十年中，世界在线活动显着增加。因此，大规模数据存储/通信目前是并将继续是一个巨大的工程挑战。我们在这一领域的活动将培训需求量很大并参与影响非常大的工程活动的总部人员。