CRII: CNS: Toward a Scalable Geo-distributed Data Analytics System

CRII：CNS：迈向可扩展的地理分布式数据分析系统

• 批准号: 2153422
• 负责人: Kwangsung Oh
• 金额: $ 17.22万
• 依托单位: University of Nebraska at Omaha
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-07-01 至 2024-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2153422&HistoricalAwards=false
• 关键词: CRII; CNS; Toward; Scalable; Geo

This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2).Many applications that automate people's daily activities rely on the cloud to serve users around the globe. For example, latency-sensitive services, such as Netflix, Uber, and Airbnb, utilize cloud resources in Amazon’s geo-distributed data centers instead of managing their own data centers. These cloud application providers must analyze highly diffuse, large-scale data, including users’ activities logs and system logs — i.e., geo-distributed analytics (GDA). GDA tasks affect user experiences, system health, and operating cost; they must be completed in a timely and cost-efficient manner. However, achieving this is challenging due to the complexities of heterogeneous cloud resources and unpredictable changes in the cloud environments and applications. This project aims to design and implement a novel GDA system that determines and manages the optimal cloud resource configurations, with consideration of the wide-area network (WAN) and diverse compute resources. In addition, the system will make task scheduling decisions to process GDA queries in a timely and cost-efficient manner considering available cloud resources, input data locations, application goals, and query execution dynamics. Thus, GDA applications will be able to achieve cost-performance goals with minimal effort, while addressing and overcoming the challenges above. The successful completion of this project will advance the understanding of (1) how GDA workloads can be accurately predicted using historical data, and (2) how diverse and heterogeneous cloud resources can be optimally allocated and efficiently utilized in GDA. A design and prototype implementation will be publicly available in the form of an open-source system. The outcomes of this research will allow researchers and software developers across several scientific, commercial, and social domains to easily run and evaluate diverse GDA algorithms. The research outcomes of this project will also be integrated into the university course curriculum for both undergraduate- and graduate-level students. This project will also open diverse opportunities that broaden participation and diversity in STEM education.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该奖项的全部或部分资金来源于《2021 年美国救援计划法案》（公法 117-2）。许多自动化人们日常活动的应用程序都依赖云为全球用户提供服务，例如延迟敏感的服务。 Netflix、Uber 和 Airbnb 等公司利用亚马逊地理分布式数据中心的云资源，而不是管理自己的数据中心。这些云应用提供商必须分析高度分散的大规模数据，包括用户的活动日志和数据。系统日志——即地理分布式分析 (GDA) 任务会影响用户体验、系统运行状况和运营成本；但是，由于其复杂性，实现这一点具有挑战性。该项目旨在设计和实现一种新颖的 GDA 系统，该系统可以考虑广域网 (WAN) 和多样化的计算资源，确定和管理最佳的云资源配置。在此外，系统将考虑可用的云资源、输入数据位置、应用程序目标和查询执行动态，做出任务调度决策，以便及时且经济高效地处理 GDA 查询。因此，GDA 应用程序将能够实现性价比。以最小的努力实现目标，同时解决和克服上述挑战，该项目的成功完成将加深人们对以下方面的理解：(1) 如何使用历史数据准确预测 GDA 工作负载，以及 (2) 云资源的多样性和异构性。在 GDA 中得到优化分配和有效利用。这项研究的设计和实现将以开源系统的形式公开，这将使多个科学、商业和社会领域的研究人员和软件开发人员能够轻松运行和评估各种 GDA 算法的结果。该项目还将纳入本科生和研究生的大学课程中。该项目还将提供多种机会，扩大 STEM 教育的参与度和多样性。该奖项是 NSF 的法定使命，并被认为值得通过以下方式获得支持。利用基金会的智力优势和更广泛的评估影响审查标准。

项目成果

Smartpick: Workload Prediction for Serverless-enabled Scalable Data Analytics Systems

Smartpick：支持无服务器的可扩展数据分析系统的工作负载预测

• DOI: 10.1145/3590140.3592850
• 发表时间: 2023-07-25
• 期刊: Proceedings of the 24th International Middleware Conference
• 作者: Anshuman Mohapatra;Kwangsung Oh
• 通讯作者: Kwangsung Oh