Big Data Processing in Heterogeneous Cloud Environments

异构云环境中的大数据处理

• 批准号: RGPIN-2018-06847
• 负责人: Shea, Ryan
• 金额: $ 1.68万
• 依托单位: Simon Fraser University
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2019
• 资助国家: 加拿大
• 起止时间: 2019-01-01 至 2020-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=679166
• 关键词: Big; Data; Processing; Heterogeneous; Cloud

This proposal focuses on the intersection of Big Data and Cloud Computing, using advanced monitoring to allow the analysis and optimization of these complex interactions. New techniques can be developed, which combine concepts from both the Big Data and Cloud Computing fields and optimizes them for both performance and energy consumption. We will study how Big Data problems map to Cloud Computing systems with a focus on scaling and workload aware design. For example, we have an opportunity to improve Big Data workload by using virtual machine placement, in order to avoid congestion in the data centre. However, intelligent workload placements using large scale cyber and physical monitoring requires in depth exploration. Our research program will focus on how to best augment and improve these technologies by combining workload awareness with an awareness of the infrastructure and physical world.******Although the Cloud has been a powerful concept and tool, there are still vast improvements that can be made. One such advancement will come in the form of continued augmentation of the existing cloud with new processing capabilities. We will study and improve these complex heterogeneous environments with the goal of improving performance and lowering energy consumption. An avenue that needs further exploration is the deployment of GPGPU (general-purpose computing on graphics processing units) and FPGA (field programmable gate array) in these complex environments. The wide deployment of these specialized components will greatly expand the potential of today's cloud offerings, not only from a user's perspective but also from that of a cloud provider. Users will be able to improve their data processing applications, for example accelerating Spark by utilizing a GPU or FPGA. Cloud providers can reduce their service hosting overhead by using these specialized devices to perform important infrastructure critical tasks such as processing packets in the virtual network. However, the presence of multiple processing engines such as the CPU, GPU, and FPGA presents an interesting problem for both the cloud user and cloud provider. The mapping of workloads to these devices to maximize the performance of the cloud becomes paramount. Of equal importance, which mapping of tasks results in lower energy consumption and heat production of the physical cloud systems.******Research collaboration with industrial and government partners will play a pivotal role in my research program. For example, we are working with Compute Canada to help optimize and develop the next generation large scale computing platforms and applications by studying the Cedar super computer. Through monitoring, analysis, and modeling we will look at performance and issues in these large scale systems and devise optimizations. The development and improvement of these advanced platforms will continue to keep Canada at the forefront of research computing.

该建议着重于大数据和云计算的交集，使用高级监视，允许对这些复杂相互作用进行分析和优化。可以开发新技术，从大数据和云计算字段中结合了概念，并为性能和能源消耗提供了优化。我们将研究大数据问题如何映射到云计算系统，重点是缩放和工作负载意识设计。例如，我们有机会使用虚拟机放置来改善大数据工作量，以避免数据中心的拥塞。但是，使用大型网络和物理监测的智能工作负载需要深入探索。我们的研究计划将集中于如何通过将工作负载意识与对基础架构和物理世界的认识相结合。这样的进步将以新的处理能力的持续增强形式出现。我们将研究和改善这些复杂的异质环境，以提高性能和降低能耗。需要进一步探索的途径是在这些复杂环境中的GPGPU（图形处理单元上的通用计算）和FPGA（现场可编程门阵列）的部署。这些专业组件的广泛部署将大大扩大当今云产品的潜力，不仅是从用户的角度来看，而且还从云提供商的角度来扩大潜力。用户将能够改善其数据处理应用程序，例如通过使用GPU或FPGA加速火花。云提供商可以通过使用这些专业设备来执行重要的基础结构，例如虚拟网络中的处理数据包来减少其服务托管开销。但是，CPU，GPU和FPGA等多个处理引擎的存在为云用户和云提供商带来了一个有趣的问题。将工作负载映射到这些设备以最大化云的性能变得很重要。同等重要的是，任务的映射会导致物理云系统的能源消耗和热量产生。******与工业和政府合作伙伴的研究合作将在我的研究计划中发挥关键作用。例如，我们正在与Compute Canada合作，通过研究Cedar超级计算机来帮助优化和开发下一代大规模计算平台和应用程序。通过监视，分析和建模，我们将研究这些大规模系统中的性能和问题，并设计优化。这些高级平台的开发和改进将继续使加拿大处于研究计算的最前沿。