Application Characterization for Adaptive Computing Platform Determination for Computational and Data-Enabled Science and Engineering

计算和数据支持的科学与工程的自适应计算平台确定的应用表征

• 批准号: 1404981
• 负责人: Haiying Shen
• 金额: $ 69.98万
• 依托单位: Clemson University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2014
• 资助国家: 美国
• 起止时间: 2014-05-01 至 2016-10-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1404981&HistoricalAwards=false
• 关键词: Application; Characterization; Adaptive; Computing; Platform

Traditional high performance computing clusters (HPCC) and Hadoop clusters are both important platforms in computational and data-enabled science and engineering (CDS&E). Hadoop MapReduce is typically effective in large-scale data analysis while traditional HPC is more commonly employed in computational problems (petabytes vs. petaflops). HPCC can include a shim layer that allows Hadoop MapReduce to access HPC storage (denoted by Hadoop+HPCC), local storage (denoted by Hadoop), and a combination of both (denoted by Hadoop/HPCC). This project aims to identify the characteristics of various CDS&E MapReduce computational tasks, and adaptively determine the best platform (Hadoop, Hadoop+HPCC and Hadoop/HPCC) for individual applications based on their characteristics, and also optimally arrange data placement between local storage and dedicated remote storage, given performance objectives and system cost metrics. Broader impacts include critical insights into the suitability of different computing platforms to different CDS&E applications, and a more advanced HPC system. Research results will be disseminated through technology transfer to industry partners, via publication in peer review journals, and in software releases. Results will also serve as catalyst for research in cyberinfrastructure, which serves the CDS&E fields. This project will provide thorough training of students and collaborative research opportunities for participating Clemson graduates, undergraduates, faculty, and K-12 students. Results will be integrated into courses taught by the PIs. The PIs will recruit new students, particularly those from underrepresented groups, to undertake the study of a STEM discipline.

传统的高性能计算集群（HPCC）和Hadoop集群都是计算和数据支持的科学与工程（CDS＆E）的重要平台。 Hadoop MapReduce 通常在大规模数据分析中有效，而传统 HPC 更常用于计算问题（PB 与 petaflops）。 HPCC 可以包括垫片层，该垫片层允许 Hadoop MapReduce 访问 HPC 存储（由 Hadoop+HPCC 表示）、本地存储（由 Hadoop 表示）以及两者的组合（由 Hadoop/HPCC 表示）。该项目旨在识别各种CDS&E MapReduce计算任务的特征，并根据其特征自适应地确定各个应用程序的最佳平台（Hadoop、Hadoop+HPCC和Hadoop/HPCC），并优化安排本地存储和专用存储之间的数据放置。给定性能目标和系统成本指标的远程存储。更广泛的影响包括对不同计算平台对不同 CDS&E 应用程序的适用性的重要见解，以及更先进的 HPC 系统。研究成果将通过向行业合作伙伴转让技术、在同行评审期刊上发表以及在软件版本中传播。结果还将成为服务于 CDS&E 领域的网络基础设施研究的催化剂。该项目将为参与的克莱姆森大学毕业生、本科生、教师和 K-12 学生提供全面的学生培训和合作研究机会。结果将被纳入 PI 教授的课程中。 PI 将招募新学生，特别是来自代表性不足群体的学生，进行 STEM 学科的学习。