CIF:Small:Collaborative Research:Efficient Codes and their Performance Limits for Distributed Storage Systems

CIF：小：协作研究：分布式存储系统的高效代码及其性能限制

• 批准号: 1421848
• 负责人: P. Vijay Kumar
• 金额: $ 16.27万
• 依托单位: University of Southern California
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2014
• 资助国家: 美国
• 起止时间: 2014-09-01 至 2019-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1421848&HistoricalAwards=false
• 关键词: CIF; Small; Collaborative; Research; Efficient

The explosive growth of data being generated and collected has rekindled interest in efficient means of storing such data. Large data centers and distributed storage systems have become more widespread, playing an ever-increasing role in our everyday computational tasks. While a data center should never lose data, disk failures occur on a daily basis as confirmed by the industry statistics. Methods and ideas from error correcting codes developed in this project enable the system to provide better guarantees against data loss as well as to reduce the amount of data that needs to be moved in order to enable recovery of information lost due to disk failures. Another related goal of this project is the reduction of storage overhead needed to support the recovery procedures. These goals are accomplished by relying on algebraic methods of constructing the data encoding procedures as well as on novel algorithms of data exchange and recovery. Overall the research performed in the course of this project contributes to the development of more efficient data management procedures in large-scale distributed storage systems.This project puts forward new algebraic procedures for data encoding and recovery that enables one to achieve tradeoff between overhead and repair bandwidth based on the concept of local recovery. The project studies both the case of recovering from a single disk loss, which is the most frequent problem in systems, as well as from the failure of multiple disks, addressing the problem of correcting one erasure as well as multiple erasures in data encoding. New bounds on the distance of codes with the locality requirement derived in this research are attained with new constructions of optimal locally recoverable codes equipped with simple recovery procedures. The project also addresses the problem of simultaneous recovery of data from multiple locations, enhancing data availability in large-scale distributed storage systems which are a key backbone component of the 21st century economy.

正在生成和收集的数据的爆炸性增长重新激发了对存储此类数据的有效手段的兴趣。 大型数据中心和分布式存储系统已经变得更加普遍，在我们的日常计算任务中发挥了不断增长的作用。 尽管数据中心绝不应该丢失数据，但磁盘故障每天都会发生，如行业统计数据所证实。来自此项目中开发的错误校正代码的方法和想法使系统能够更好地保证数据丢失数据，并减少需要移动的数据量，以便能够恢复由于磁盘故障而丢失的信息。该项目的另一个相关目标是减少支持恢复程序所需的存储开销。这些目标是通过依靠代数方法来构建数据编码过程以及新颖的数据交换和恢复算法来实现的。总的来说，在该项目过程中进行的研究有助于开发大规模分布式存储系统中更有效的数据管理程序。本项目为数据编码和恢复提供了新的代数程序，使人们能够根据本地回收的概念实现高架和维修带宽之间的权衡。 该项目研究了从单个磁盘损耗中恢复的情况，这是系统中最常见的问题，也是多个磁盘的故障，解决了纠正一个擦除以及数据编码中多个擦除的问题。在本研究中得出的当地要求的代码距离上的新界限是通过配备简单恢复程序的最佳本地可回收代码的新结构来实现的。该项目还解决了从多个位置同时恢复数据的问题，从而增强了大规模分布式存储系统的数据可用性，这是21世纪经济的关键骨干组成部分。