III: Small: Data Analysis in the Cloud with Guaranteed and Explainable Performance

III：小型：云端数据分析，性能有保证且可解释

• 批准号: 1524535
• 负责人: Magdalena Balazinska
• 金额: $ 50万
• 依托单位: University of Washington
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2015
• 资助国家: 美国
• 起止时间: 2015-09-01 至 2019-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1524535&HistoricalAwards=false
• 关键词: III; Small; Data; Analysis; Cloud

Increasingly many users have access to large datasets that they need to analyze. Astronomers, oceanographers, and other domain scientists rely on data analysis for their science. Journalists may want to analyze data to use in their articles. Over the past several years, cloud service providers have been offering an increasingly large selection of data management services for data analytics (e.g., Amazon Elastic MapReduce or Google BigQuery). Cloud services provide a seamless access to powerful data analysis tools, often directly through the browser. Too many services, however, remain too close to the traditional mode of operating a database management system. They reveal too much information about their internal architecture and deployment: Users are required to reason at the level of service instances, instance types, and gigabytes processed. As a result, users today must be data management experts to choose between these services and leverage them in a cost-effective manner. This project will develop new data management techniques that will enable cloud service providers to isolate users from the details of their service internals while offering the ability to trade off price and performance. The project will further develop tools to explain performance and help users re-write their queries to improve it.More specifically, the project will develop new approaches to (1) predict not only the query runtime but whether a query is likely to execute slower than estimated due to failures, skew, cardinality estimation errors, or contention; (2) guarantee query runtimes by dynamically changing both the resources allocated to a query and its failure-handling and skew-handling mechanisms as needed; (3) post specific slowdown factors in case of heavy load and guarantee them through novel scheduling algorithms; and (4) explain query performance and suggest rewrites in a way that does not require users to understand query plans. The project will implement all of the algorithms in the open source Myria cloud data management system (and service) recently developed and in continuous operation at the University of Washington.For further information see the project web site at: http://cloudperf.cs.washington.edu

越来越多的用户可以访问他们需要分析的大型数据集。天文学家，海洋学家和其他领域科学家依靠其科学的数据分析。记者可能想分析以在其文章中使用的数据。在过去的几年中，云服务提供商一直为数据分析提供越来越多的数据管理服务（例如，亚马逊弹性MapReduce或Google Bigquery）。通常直接通过浏览器直接通过浏览器，云服务可无缝访问强大的数据分析工具。但是，太多的服务与操作数据库管理系统的传统方式太接近了。他们揭示了有关其内部体系结构和部署的太多信息：需要用户在处理的服务实例，实例类型和千兆字节上进行推理。结果，今天的用户必须是数据管理专家，以在这些服务之间进行选择，并以具有成本效益的方式利用它们。该项目将开发新的数据管理技术，这将使云服务提供商能够将用户与服务内部的详细信息隔离，同时提供交易价格和绩效的能力。该项目将进一步开发工具来解释性能并帮助用户重新编写其查询以改进它。更具体地说，该项目将开发新的方法来（1）不仅预测查询运行时，还可以预测查询是否可能比由于失败，偏斜，心脏限额估计错误或竞争而导致的估计值较慢； （2）通过动态更改分配给查询的资源及其故障处理和偏向处理机制来保证查询运行时； （3）在重负荷的情况下，在特定的放缓因子上发布特定的放缓因子，并通过新颖的调度算法来保证它们； （4）解释查询性能，并建议以不需要用户了解查询计划的方式重写。 该项目将在最近开发的开源Myria Cloud数据管理系统（和服务）中实施所有算法，并在华盛顿大学连续运行。有关更多信息，请参见项目网站：http：//cloudperf.cs.washington.edu