Intelligent Internet-Scale Multimedia Storage and Delivery

智能互联网规模多媒体存储和传输

• 批准号: 436170-2013
• 负责人: Niu, Di
• 金额: $ 1.82万
• 依托单位: University of Alberta
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2018
• 资助国家: 加拿大
• 起止时间: 2018-01-01 至 2019-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=660993
• 关键词: Intelligent; Internet; Scale; Multimedia; Storage

Video services, such as Youtube and Netflix, are enormously popular on the Internet nowadays. With the advent of datacenters, cloud computing and increased Internet bandwidth, video delivery is experiencing a major architectural change. As a typical Internet-scale video delivery system, Netflix hosts its content in Amazon Simple Storage Service (S3), which organizes a storage system spanning geographically distributed datacenters. The videos are then delivered to end users through multiple Content Delivery Networks (CDNs) as edge servers. Our grand goal in this proposal is to build intelligence into future video storage and delivery systems at the Internet scale by incorporating data-intensive decision making in computer networking. The proposed system will utilize various types of data collected from the network, including system traces, logs, user reports and even social information, for workload prediction and performance inference. The processed data will then be used to assist content placement, resource management, data transfers and load routing, which can be modelled as data-intensive optimization problems, at multiple levels in the video distribution hierarchy. The key challenges to be addressed include how to learn useful information from data, how to approach various combinatorial optimization problems required by the application, and especially, how to efficiently solve such data-intensive optimizations on-the-fly to satisfy application requirements. In order to scale to very large systems and a large amount of input data, we will particularly investigate novel distributed and parallel algorithms and computation paradigms, which either lead to decentralized decisions in the network or efficient parallel computation through Hadoop and cloud computing services. The proposed system will eventually integrate large-scale computer networking and traffic engineering with data-intensive computing and decision making at the application level. Our study will be based on a large amount of data collected from various video delivery and CDN companies. We aim to test our approaches in real network experiments involving PlanetLab nodes, Amazon EC2 datacenters and server clusters, depending on resource availability.**

Youtube 和 Netflix 等视频服务如今在互联网上非常流行。随着数据中心、云计算和互联网带宽的增加，视频传输正在经历重大的架构变化。作为典型的互联网规模视频传输系统，Netflix 将其内容托管在 Amazon Simple Storage Service (S3) 中，该服务组织了一个跨越地理分布的数据中心的存储系统。然后，视频通过作为边缘服务器的多个内容交付网络 (CDN) 交付给最终用户。我们在此提案中的宏伟目标是通过将数据密集型决策纳入计算机网络中，将智能构建到互联网规模的未来视频存储和传输系统中。所提出的系统将利用从网络收集的各种类型的数据（包括系统跟踪、日志、用户报告甚至社交信息）来进行工作负载预测和性能推断。然后，处理后的数据将用于协助内容放置、资源管理、数据传输和负载路由，这些可以在视频分发层次结构的多个级别上建模为数据密集型优化问题。要解决的关键挑战包括如何从数据中学习有用的信息，如何处理应用程序所需的各种组合优化问题，特别是如何有效地即时解决此类数据密集型优化以满足应用程序需求。为了扩展到非常大的系统和大量的输入数据，我们将特别研究新颖的分布式并行算法和计算范式，它们要么导致网络中的去中心化决策，要么通过 Hadoop 和云计算服务实现高效的并行计算。所提出的系统最终将把大规模计算机网络和交通工程与应用层面的数据密集型计算和决策相结合。我们的研究将基于从各个视频传输和 CDN 公司收集的大量数据。我们的目标是在涉及 PlanetLab 节点、Amazon EC2 数据中心和服务器集群的真实网络实验中测试我们的方法，具体取决于资源可用性。**