EAGER: Collaborative Research: Scaling Up Discriminative Learning for Natural Language Understanding and Translation

EAGER：协作研究：扩大自然语言理解和翻译的判别学习

• 批准号: 1656051
• 负责人: Liang Huang
• 金额: $ 9.04万
• 依托单位: Oregon State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2015
• 资助国家: 美国
• 起止时间: 2015-08-27 至 2018-12-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1656051&HistoricalAwards=false
• 关键词: EAGER; Collaborative; Research; Scaling; Up

This EArly Grant for Exploratory Research aims to improve automatic understanding of natural language by machines, and automatic translation between languages such as Chinese and English. In the realm of understanding, the project develops methods for syntactically and semantically analyzing, or parsing, sentences. Improved parsing can help in accessing the enormous amount of information available in unstructured text on the web and in databases of newspapers and scanned books. Improved translation between languages increases opportunities for trade as well as for dissemination of information generally between nations and cultures. Machine translation is widely used today despite its generally poor quality, and any improvement in quality will improve access to information for millions of people. This project aims to exploit the power of machine learning algorithms that are designed to discriminate between correct and incorrect outputs by numerically optimizing mathematical functions that are defined in terms of the data available for training. Discriminative structured prediction algorithms have witnessed great success in the field of natural language processing (NLP) over the past decade, generally surpassing their generative counterparts. However, there remain two major problems which prevent discriminative methods from scaling to very large datasets: first, they typically assume exact search (over a prohibitively large search space), which is rarely possible in practice for problems such as parsing and translation. Secondly, they normally assume the data is completely annotated, whereas many naturally occurring datasets are only partially annotated: for example a parallel text in machine translation includes the source and target sentence pairs but not the derivation between them. As a result of these two problems, the current methods are not taking full advantage of the enormous and ever increasing amount of text data available to us.This EArly Grant ofr Exploratory Research (EAGER) aims to: - Develop a linear-time structured learning framework specifically tailored for inexact search, which hopefully retains theoretical properties of structured learning (e.g. convergence) under exact search. - Extend this framework to handle latent variables, such as derivations in machine translation, syntactic structures in semantic parsing, and semantic representations in question answering. If the exploratory extension to latent variable frameworks is sucessful, it will enable longer-term research to: - Apply these efficient learning algorithms to discriminative training of machine translation systems over the entire training dataset rather than only on a small development set. - Apply these efficient learning algorithms to discriminative training for syntactic and semantic parsing, with the goal of scaling up semantic parsing to enable web-scale knowledge extraction.

这项早期探索性研究资助旨在提高机器对自然语言的自动理解，以及中文和英文等语言之间的自动翻译。在理解领域，该项目开发了对句子进行句法和语义分析或解析的方法。改进的解析可以帮助访问网络上非结构化文本以及报纸和扫描书籍数据库中的大量信息。语言之间翻译的改进增加了贸易机会以及国家和文化之间的信息传播机会。尽管机器翻译的质量普遍较差，但如今机器翻译仍被广泛使用，而质量的任何改进都将改善数百万人获取信息的机会。 该项目旨在利用机器学习算法的强大功能，该算法旨在通过对根据可用于训练的数据定义的数学函数进行数值优化来区分正确和错误的输出。 过去十年，判别式结构化预测算法在自然语言处理（NLP）领域取得了巨大成功，总体上超越了生成式预测算法。然而，仍然存在两个主要问题阻碍判别性方法扩展到非常大的数据集：首先，它们通常假设精确搜索（在非常大的搜索空间上），这在实践中对于解析和翻译等问题很少是可能的。其次，他们通常假设数据已完全注释，而许多自然存在的数据集仅部分注释：例如，机器翻译中的并行文本包括源句和目标句对，但不包括它们之间的派生。由于这两个问题，当前的方法没有充分利用我们可用的大量且不断增加的文本数据。这项早期探索性研究资助 (EAGER) 旨在： - 开发线性时间结构化学习专门为不精确搜索量身定制的框架，希望保留精确搜索下结构化学习的理论特性（例如收敛）。 - 扩展该框架以处理潜在变量，例如机器翻译中的推导、语义解析中的句法结构以及问答中的语义表示。 如果对潜变量框架的探索性扩展成功，它将使得长期研究能够： - 将这些高效的学习算法应用于整个训练数据集上的机器翻译系统的判别训练，而不仅仅是在小型开发集上。 - 将这些高效的学习算法应用于句法和语义解析的判别训练，目标是扩大语义解析以实现网络规模的知识提取。

项目成果

Breaking the Beam Search Curse: A Study of (Re-)Scoring Methods and Stopping Criteria for Neural Machine Translation

打破束搜索魔咒：神经机器翻译（重新）评分方法和停止标准的研究

• DOI: 10.18653/v1/d18-1342
• 发表时间: 2018-08-28
• 期刊: ArXiv
• 作者: Yilin Yang;Liang Huang;Mingbo Ma
• 通讯作者: Mingbo Ma

Multi-Reference Training with Pseudo-References for Neural Translation and Text Generation

用于神经翻译和文本生成的伪参考多参考训练

• DOI: 10.18653/v1/d18-1357
• 发表时间: 2018-08-28
• 期刊: ArXiv
• 作者: Renjie Zheng;Mingbo Ma;Liang Huang
• 通讯作者: Liang Huang

Speeding Up Neural Machine Translation Decoding by Cube Pruning

通过立方体剪枝加速神经机器翻译解码

• DOI: 10.18653/v1/d18-1460
• 发表时间: 2018-09-09
• 期刊: ArXiv
• 作者: Wen Zhang;Liang Huang;Yang Feng;Lei Shen;Qun Liu
• 通讯作者: Qun Liu

Learning to Stop in Structured Prediction for Neural Machine Translation

神经机器翻译结构化预测中的学习停止

• DOI: 10.18653/v1/n19-1187
• 发表时间: 2019-04-01
• 作者: Mingbo Ma;Renjie Zheng;Liang Huang
• 通讯作者: Liang Huang

Ensemble Sequence Level Training for Multimodal MT: OSU-Baidu WMT18 Multimodal Machine Translation System Report

多模态机器翻译的集成序列级训练：OSU-百度 WMT18 多模态机器翻译系统报告

• DOI: 10.18653/v1/w18-6443
• 发表时间: 2018-08-31
• 作者: Renjie Zheng;Yilin Yang;Mingbo Ma;Liang Huang
• 通讯作者: Liang Huang