SGER: Scaling up unsupervised grammar induction

SGER：扩大无监督语法归纳

• 批准号: 0836431
• 负责人: Noah Smith
• 金额: --
• 依托单位: Carnegie-Mellon University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2008
• 资助国家: 美国
• 起止时间: 2008-07-01 至 2009-12-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0836431&HistoricalAwards=false
• 关键词: SGER; Scaling; up; unsupervised; grammar

This SGER project seeks to determine the scalability of computationally intensive, iterative statistical learning algorithms on a MapReduce architecture. Such algorithms underlie much research in natural language processing, yet their scalability to even moderately large training datasets (text corpora) has been under-explored. On the surface, scaling to more data appears to be a good fit for the MapReduce paradigm, and this exploratory project aims to identify whether such algorithms benefit from more data and more complex data than used in prior work. A special emphasis is given to unsupervised learning algorithms, such as the Expectation-Maximization algorithm, which have been widely studied on small problems and rarely studied on large ones. The technique is applicable to many other methods, as well.At the same time, the project seeks to explore how to leverage supercomputers and MapReduce to make these learning algorithms faster, permitting a faster research cycle. Concretely, the "E step" (or itsanalogue) is the most computationally demanding part of an iteration, but the standard assumption that the training data are independently and identically distributed permits parallelization. To the extent that this parallelization is affected by network and input-output overhead, each iteration of training may be made faster, perhaps reducing training time from days or weeks to hours. This project explores this tradeoff and others like it.This work leverages a resource donated by Yahoo for use by the PI's research group: a 4,000-node supercomputer running Hadoop (an open-source implementation of MapReduce).

该SGER项目旨在确定MapReduce体系结构上计算密集型，迭代统计学习算法的可伸缩性。 这种算法在自然语言处理方面进行了很多研究，但是它们对中等大型培训数据集（文本语料库）的可扩展性却尚未探索。 从表面上看，将扩展到更多数据似乎非常适合MapReduce范式，并且该探索性项目旨在确定此类算法是否从更多的数据和更复杂的数据中受益于先前的工作。 特别强调了无监督的学习算法，例如期望最大化算法，这些算法已经广泛研究了小问题，很少对大型问题进行研究。 该技术也适用于许多其他方法。在同一时间，该项目试图探索如何利用超级计算机和MAPREDUCE，以使这些学习算法更快，从而更快地进行研究周期。 具体而言，“ E步骤”（或ISANALOGUE）是迭代中计算最高的部分，但是训练数据是独立且分布相同的允许并行化的标准假设。 在某种程度上，这种并行化受网络和投入输出开销的影响，培训的每次迭代都可以更快地进行，也许将训练时间从几天或几周减少到数小时。 该项目探讨了这种折衷方案和其他类似的折衷。这项工作利用了Yahoo捐赠的资源供Pi的研究小组使用：4,000个节点超级计算机运行Hadoop（MapReduce的开源实现）。