Advanced coding solutions for large-scale data storage & communication

适用于大规模数据存储的高级编码解决方案

• 批准号: RGPIN-2017-04119
• 负责人: Ardakani, Masoud
• 金额: $ 2.4万
• 依托单位: University of Alberta
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2019
• 资助国家: 加拿大
• 起止时间: 2019-01-01 至 2020-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=684518
• 关键词: Advanced; coding; solutions; large; scale

Motivation***The global IP traffic is growing rapidly due to increased demand for online services such as cloud storage, video on demand, live streaming, and file sharing. These online services, which are responsible for the bulk of the Internet traffic, heavily rely on distributed storage systems (DSS). Hence, around the world, the number and size of DSSs have rapidly increased in recent years. In fact, DSSs are already among the top contributors of carbon emission in the modern world. In the US, for example, they are only second to the airline industry and expected to be the top contributor by 2020. Thus, research on efficient data storage techniques for DSSs has recently attracted a lot of interest both in academia and in industry.***Background***Hardware and software failures in a DSS result in data loss and unavailability. Naturally, many service providers keep multiple copies of data for reliability and availability. The downside, of course, is the huge storage overhead (typically 200%) and the associated maintenance costs. As a solution, recently, error correction coding is suggested for DSSs. Unfortunately, most existing error correction codes cannot be used in these large-scale applications, because the repair traffic that they create is beyond the network capacity of the DSSs. Hence, a strong need for more efficient codes, tailored for large-scale applications, is felt. ***Goals ***In this research program, using our past knowledge of modern coding and our recent activities on coding for DSSs, we will develop efficient and scalable error correction coding solutions for distributed storage systems. We will also seek theoretical results on the fundamental relations among different performance measures (such as coding overhead, code reliability, repair traffic, update complexity, etc.). ***There are a large number of high-impact open problems in this area all suitable for training HQPs.***Impact***This research proposal follows several goals that are exceedingly important to both practitioners and theorists in the area of coding and information theory. Improving the efficiency of large-scale data storage not only improves our online experience, but also significantly reduces the carbon footprint of DSSs. We hope our results throughout this work and our contribution with other Canadian researchers turn Canada into a major scientific center in this field, which can technically and scientifically support any data center located in Canada. In fact, since cooling is a major cost in data centers, Canada because of its colder climate and its safety is considered as an ideal place to host data centers. ***In the past decade, the world has seen a significant increase in online activity. As a result, large-scale data storage/communication currently is and will continue to be a big engineering challenge. Our activity in this field will train HQPs that will be in high demand and involved in very high impact engineering activities.

动机***由于对在线服务的需求增加，例如云存储，按需视频，直播和文件共享，全球IP流量正在迅速增长。这些在线服务负责大部分互联网流量，严重依赖分布式存储系统（DSS）。因此，在世界范围内，近年来，DSS的数量和大小迅速增加。实际上，DSS已经成为现代世界中碳排放的主要贡献者之一。例如，在美国，它们仅次于航空业，预计到2020年将成为最大的贡献者。因此，对DSSS的有效数据存储技术的研究最近在学术界和行业中引起了很多兴趣。自然，许多服务提供商保留了多个数据副本，以提供可靠性和可用性。当然，缺点是巨大的存储空间（通常为200％）和相关的维护成本。作为解决方案，最近建议针对DSSS进行误差校正编码。不幸的是，大多数现有的错误校正代码无法在这些大规模应用中使用，因为它们创建的维修流量超出了DSS的网络容量。因此，人们认为，对更有效的代码的强烈需求是为大规模应用量身定制的。 ***目标***在本研究计划中，使用我们过去对现代编码的知识以及我们最近针对DSSS编码的活动，我们将为分布式存储系统开发有效且可扩展的错误校正编码解决方案。我们还将寻求有关不同绩效指标之间基本关系（例如编码开销，代码可靠性，维修流量，更新复杂性等）之间基本关系的理论结果。 ***在该领域有很多适合培训HQP的高影响力的开放问题。提高大规模数据存储的效率不仅可以改善我们的在线体验，而且大大降低了DSSS的碳足迹。我们希望我们在整个工作中的结果以及与加拿大其他研究人员的贡献将加拿大变成该领域的一个主要科学中心，从技术和科学上讲，该中心可以支持加拿大的任何数据中心。实际上，由于冷却是加拿大数据中心的主要成本，因为其气候较冷，其安全性被认为是托管数据中心的理想场所。 ***在过去的十年中，全世界的在线活动显着增加。结果，当前并且将继续是一个巨大的工程挑战，目前将继续进行大规模的数据存储/通信。我们在该领域的活动将训练高需求并参与非常高影响力工程活动的基质水平。