RI: Small: Collaborative Research: Statistical Learning of Language Universals

RI：小型：协作研究：语言共性的统计学习

• 批准号: 1116676
• 负责人: Benjamin Snyder
• 金额: $ 34万
• 依托单位: University of Wisconsin-Madison
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2011
• 资助国家: 美国
• 起止时间: 2011-08-01 至 2015-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1116676&HistoricalAwards=false
• 关键词: RI; Small; Collaborative; Research; Statistical

As modern technology infrastructure spreads throughout the world, the quantity of electronic text, written in hundreds of different languages, continues to grow in size and diversity. Building effective information retrieval, extraction, and translation systems across this vast array of languages currently requires time-consuming and expensive linguistic annotations for each language. Generic, fully unsupervised, methods are unlikely to provide a language independent solution to this problem.Focusing on part-of-speech prediction, this project undertakes a novel approach, combining elements of supervised and unsupervised learning without assuming any specific knowledge of the target language. Instead of treating individual languages as closed systems, language-independent "universals" are statistically estimated from dozens of languages for which annotated corpora exist, and these learned universals are used to predict the part-of-speech categories of unannotated languages. At the heart of the project is a data-driven exploration of language-independent corpus characteristics that relate cross-lingual linguistic categories to surface statistics of text. These learned patterns are incorporated into expressive structured prediction models using novel approximate learning and inference methods developed by the Principal Investigators of the project.Of the world?s spoken languages, hundreds are at risk of immediate extinction and thousands more are likely to disappear over the coming decades. By facilitating the rapid creation of language-independent linguistic analysis tools, the technology developed under this project has the potential to revolutionize the documentation of endangered languages. In the long-term, this research direction will also help realize the full social benefits of the global technology infrastructure by creating intelligent text processing tools for hundreds of low-resource languages.

随着现代技术基础设施在全球范围内传播，用数百种不同语言编写的电子文本数量在规模和多样性上不断增长。当前，跨这种大量语言构建有效的信息检索，提取和翻译系统需要每种语言的时间耗时且昂贵的语言注释。 通用，完全无监督的方法不太可能为此问题提供独立的语言解决方案。专注于言论部分的预测，该项目采用了一种新颖的方法，将受到监督和无监督学习的要素结合在一起，而无需假设目标语言任何特定的知识。 从统计上估计存在带注释的语料库的语言，而不是将单个语言视为封闭的系统，而是统计的“普遍性”，并且这些学识渊博的普遍性用于预测未经通知语言的语言类别。该项目的核心是对与语言无关的语料库特征进行数据驱动的探索，该探索将跨语性语言类别与文本的表面统计数据相关联。这些学识渊博的模式使用该项目的主要研究人员开发的新颖近似学习和推理方法将这些模式纳入了表达性结构化的预测模型中，全世界的口语语言，数百种立即灭绝的风险，而数以千计的人可能会消失。未来几十年。通过促进与语言无关的语言分析工具的快速创建，该项目下开发的技术有可能彻底改变濒危语言的文献。从长远来看，这个研究方向还将通过为数百种低资源语言创建智能文本处理工具来帮助实现全球技术基础设施的全部社会利益。