III: Small: Data Analysis in the Cloud with Guaranteed and Explainable Performance

III：小型：云端数据分析，性能有保证且可解释

• 批准号: 1524535
• 负责人: Magdalena Balazinska
• 金额: $ 50万
• 依托单位: University of Washington
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2015
• 资助国家: 美国
• 起止时间: 2015-09-01 至 2019-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1524535&HistoricalAwards=false
• 关键词: III; Small; Data; Analysis; Cloud

Increasingly many users have access to large datasets that they need to analyze. Astronomers, oceanographers, and other domain scientists rely on data analysis for their science. Journalists may want to analyze data to use in their articles. Over the past several years, cloud service providers have been offering an increasingly large selection of data management services for data analytics (e.g., Amazon Elastic MapReduce or Google BigQuery). Cloud services provide a seamless access to powerful data analysis tools, often directly through the browser. Too many services, however, remain too close to the traditional mode of operating a database management system. They reveal too much information about their internal architecture and deployment: Users are required to reason at the level of service instances, instance types, and gigabytes processed. As a result, users today must be data management experts to choose between these services and leverage them in a cost-effective manner. This project will develop new data management techniques that will enable cloud service providers to isolate users from the details of their service internals while offering the ability to trade off price and performance. The project will further develop tools to explain performance and help users re-write their queries to improve it.More specifically, the project will develop new approaches to (1) predict not only the query runtime but whether a query is likely to execute slower than estimated due to failures, skew, cardinality estimation errors, or contention; (2) guarantee query runtimes by dynamically changing both the resources allocated to a query and its failure-handling and skew-handling mechanisms as needed; (3) post specific slowdown factors in case of heavy load and guarantee them through novel scheduling algorithms; and (4) explain query performance and suggest rewrites in a way that does not require users to understand query plans. The project will implement all of the algorithms in the open source Myria cloud data management system (and service) recently developed and in continuous operation at the University of Washington.For further information see the project web site at: http://cloudperf.cs.washington.edu

越来越多的用户可以访问他们需要分析的大型数据集。天文学家、海洋学家和其他领域的科学家依靠数据分析来进行科学研究。记者可能想要分析数据以在他们的文章中使用。在过去几年中，云服务提供商为数据分析提供了越来越多的数据管理服务选择（例如 Amazon Elastic MapReduce 或 Google BigQuery）。云服务通常可以直接通过浏览器无缝访问强大的数据分析工具。然而，太多的服务仍然过于接近操作数据库管理系统的传统模式。它们揭示了太多有关其内部架构和部署的信息：用户需要在服务实例、实例类型和处理的千兆字节级别进行推理。因此，当今的用户必须是数据管理专家，才能在这些服务之间进行选择并以经济高效的方式利用它们。该项目将开发新的数据管理技术，使云服务提供商能够将用户与其服务内部的细节隔离开来，同时提供在价格和性能之间进行权衡的能力。该项目将进一步开发工具来解释性能并帮助用户重新编写查询以改进性能。更具体地说，该项目将开发新方法来 (1) 不仅预测查询运行时间，而且预测查询是否可能执行得比由于故障、偏差、基数估计错误或争用而估计； (2) 通过根据需要动态改变分配给查询的资源及其故障处理和倾斜处理机制来保证查询运行时间； (3) 在重负载的情况下发布特定的减速因素，并通过新颖的调度算法保证它们； (4) 解释查询性能并建议以不需要用户理解查询计划的方式重写。 该项目将实现华盛顿大学最近开发并持续运行的开源 Myria 云数据管理系统（和服务）中的所有算法。有关更多信息，请参阅该项目网站：http://cloudperf.cs .washington.edu