II-EN: Hadoop NextGen Infrastructure for Heterogeneous Approaches to Data-Intensive Computing

II-EN：用于数据密集型计算异构方法的 Hadoop NextGen 基础设施

• 批准号: 1405688
• 负责人: Jimmy Lin
• 金额: $ 49.99万
• 依托单位: University of Maryland, College Park
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2014
• 资助国家: 美国
• 起止时间: 2014-08-01 至 2017-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1405688&HistoricalAwards=false
• 关键词: II; EN; Hadoop; NextGen; Infrastructure

The ability for organizations to process enormous quantities of data and to extract insights from those data has revolutionized commerce and science. This phenomenon, known as "big data", is shaping the very fabric of our society. Our insatiable appetite for more data, and knowledge from the data, requires significant computational infrastructure for storage and analytical capabilities. Continued investments in infrastructure for academic researchers are vital from two perspectives: From the research perspective, the university's ability to help advance the state of the art in big data technologies is dependent on access to the right computational resources. From the educational perspective, the university's mission to train the next generation of scientists and engineers cannot be successfully accomplished without big data infrastructure that is becoming essential to their careers. The goal of this project is to provide computational resources to researchers at the University of Maryland to continuing envisioning the future of big data.The modern empirical approach to tackling many challenges in natural language processing, information retrieval, data mining, machine learning, and other related domains involves exploiting large amounts of data to learn statistical models that are able to capture characteristics of the problem. A necessary ingredient to this "big data" approach is scalable infrastructure that can distribute computations across a cluster of machines. Hadoop, the open-source implementation of MapReduce, has achieved widespread adoption as the de facto platform for data-intensive computing.Broadly speaking, MapReduce excels at large-scale content analysis in an offline, batch setting. However, this is not enough: we need a data-intensive computing platform that supports heterogeneous models of computation. Hadoop NextGen (aka YARN), provides exactly this: it allows a physical cluster to support a wide range of computational models via a generic resource allocation framework.This project supports the acquisition of a Hadoop NextGen cluster at the University of Maryland to support the following activities:1. To explore computational models beyond MapReduce, including batch/online tradeoffs in machine learning, real-time streaming computations, and graph processing.2. To sustain innovations in algorithms for content analysis as well as modeling implicit and latent relationships between heterogeneous content (text, images, graphs, etc.) at scale.3. To exploit novel hardware architectures for data-intensive computing (e.g., Graphics Processing Units and Solid State Drives).These resources will help the Laboratory for Computational Linguistics and Information Processing (CLIP) and collaborators at the University of Maryland sustain and enhance its successful record of innovation and the integration of research and education.

组织处理大量数据并从这些数据中提取见解的能力已经彻底改变了商业和科学。这种被称为“大数据”的现象正在塑造我们社会的结构。我们对更多数据以及从数据中获取知识的需求永无止境，因此需要大量的计算基础设施来存储和分析能力。 从两个角度来看，对学术研究人员基础设施的持续投资至关重要：从研究角度来看，大学帮助推进大数据技术最先进水平的能力取决于对正确计算资源的访问。从教育的角度来看，如果没有大数据基础设施，大学培养下一代科学家和工程师的使命就无法成功完成，而大数据基础设施对他们的职业生涯至关重要。 该项目的目标是为马里兰大学的研究人员提供计算资源，以继续展望大数据的未来。解决自然语言处理、信息检索、数据挖掘、机器学习等领域的许多挑战的现代实证方法相关领域涉及利用大量数据来学习能够捕获问题特征的统计模型。这种“大数据”方法的一个必要组成部分是可扩展的基础设施，它可以跨机器集群分配计算。 Hadoop 是 MapReduce 的开源实现，作为数据密集型计算事实上的平台已得到广泛采用。从广义上讲，MapReduce 擅长在离线、批处理环境中进行大规模内容分析。然而，这还不够：我们需要一个支持异构计算模型的数据密集型计算平台。 Hadoop NextGen（又名 YARN）正是提供了这一点：它允许物理集群通过通用资源分配框架支持广泛的计算模型。该项目支持在马里兰大学购买 Hadoop NextGen 集群，以支持以下功能活动：1。探索 MapReduce 之外的计算模型，包括机器学习中的批量/在线权衡、实时流计算和图形处理。2．维持内容分析算法的创新以及大规模异构内容（文本、图像、图形等）之间隐式和潜在关系的建模。3．利用新颖的硬件架构进行数据密集型计算（例如图形处理单元和固态驱动器）。这些资源将帮助计算语言学和信息处理实验室 (CLIP) 以及马里兰大学的合作者维持并提高其成功记录创新以及研究与教育的整合。