CIF:Small:Collaborative Research:Efficient Codes and their Performance Limits for Distributed Storage Systems

CIF：小：协作研究：分布式存储系统的高效代码及其性能限制

• 批准号: 1421848
• 负责人: P. Vijay Kumar
• 金额: $ 16.27万
• 依托单位: University of Southern California
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2014
• 资助国家: 美国
• 起止时间: 2014-09-01 至 2019-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1421848&HistoricalAwards=false
• 关键词: CIF; Small; Collaborative; Research; Efficient

The explosive growth of data being generated and collected has rekindled interest in efficient means of storing such data. Large data centers and distributed storage systems have become more widespread, playing an ever-increasing role in our everyday computational tasks. While a data center should never lose data, disk failures occur on a daily basis as confirmed by the industry statistics. Methods and ideas from error correcting codes developed in this project enable the system to provide better guarantees against data loss as well as to reduce the amount of data that needs to be moved in order to enable recovery of information lost due to disk failures. Another related goal of this project is the reduction of storage overhead needed to support the recovery procedures. These goals are accomplished by relying on algebraic methods of constructing the data encoding procedures as well as on novel algorithms of data exchange and recovery. Overall the research performed in the course of this project contributes to the development of more efficient data management procedures in large-scale distributed storage systems.This project puts forward new algebraic procedures for data encoding and recovery that enables one to achieve tradeoff between overhead and repair bandwidth based on the concept of local recovery. The project studies both the case of recovering from a single disk loss, which is the most frequent problem in systems, as well as from the failure of multiple disks, addressing the problem of correcting one erasure as well as multiple erasures in data encoding. New bounds on the distance of codes with the locality requirement derived in this research are attained with new constructions of optimal locally recoverable codes equipped with simple recovery procedures. The project also addresses the problem of simultaneous recovery of data from multiple locations, enhancing data availability in large-scale distributed storage systems which are a key backbone component of the 21st century economy.

生成和收集的数据的爆炸性增长重新激发了人们对存储此类数据的有效方法的兴趣。 大型数据中心和分布式存储系统已经变得越来越普遍，在我们的日常计算任务中发挥着越来越大的作用。 虽然数据中心不应该丢失数据，但行业统计数据证实，磁盘故障每天都会发生。该项目中开发的纠错码方法和思想使系统能够更好地保证数据丢失，并减少需要移动的数据量，以便恢复因磁盘故障而丢失的信息。该项目的另一个相关目标是减少支持恢复过程所需的存储开销。这些目标是通过依赖构建数据编码过程的代数方法以及数据交换和恢复的新颖算法来实现的。总体而言，该项目过程中进行的研究有助于在大规模分布式存储系统中开发更有效的数据管理程序。该项目提出了用于数据编码和恢复的新代数程序，使人们能够在开销和修复之间实现权衡基于本地恢复概念的带宽。 该项目研究了从单个磁盘丢失（系统中最常见的问题）和多个磁盘故障中恢复的情况，解决了纠正数据编码中的一次擦除和多次擦除的问题。通过配备简单恢复过程的最佳本地可恢复代码的新构造，获得了本研究中推导的具有局部性要求的代码距离的新界限。该项目还解决了从多个位置同时恢复数据的问题，增强了大规模分布式存储系统的数据可用性，而大规模分布式存储系统是 21 世纪经济的关键支柱组成部分。