CompCog: RI: Small: Human-like semantic grammar induction through knowledge distillation from pre-trained language models

CompCog：RI：Small：通过预训练语言模型的知识蒸馏进行类人语义语法归纳

• 批准号: 2313140
• 负责人: William Schuler
• 金额: $ 48.45万
• 依托单位: Ohio State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-09-01 至 2026-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2313140&HistoricalAwards=false
• 关键词: CompCog; RI; Small; Human; like

Human languages are thought to allow for an unbounded set of possible meanings to be expressed using bounded sets of rules, called grammars. These grammars assign meanings to words and compose meanings of words and phrases into larger phrases and clauses. Humans can communicate extremely precise descriptions of goals and world behaviors—and linguists know much about the logical structure of language involved in this kind of precise communication—but one of the central open questions in linguistics is how humans acquire these mechanisms. Computational models of how these grammars are learned from recorded or transcribed utterances can provide evidence that this learning can be accomplished by children without substantial innate biological biases, and these models can also provide automated tools for analysis and documentation of endangered languages, including many Indigenous American languages. Existing statistical and neural grammar learning methods can induce grammars from sentences in text corpora that predict about half of the phrases and clauses annotated by linguists; howevert this level of performance is nowhere near the accuracy of human language learners, and attempts to support this learning using image and video data have not substantially improved induction accuracy. The proposed work will instead extract statistics about logical predicates from large commercially available neural language models as a surrogate for human world knowledge so as to improve the accuracy of grammar induction.The proposed work will develop the first broad-coverage semantic grammar induction model that integrates world knowledge into the acquisition process by distilling it from large pre-trained neural language models. The world knowledge implicit in the large language models will be distilled into a matrix of predicate co-occurrence statistics using argument-specific prompts. The resulting predicate co-occurrence statistics will make no distinction between, for example, active and passive sentences, topicalized and non-topicalized sentences, or declarative and subject-auxiliary inverted sentences. This model will be used to evaluate claims about the statistical learnability of grammar. The proposed work will also continue work on developing resources for evaluating these structural models. The model and corpora collected as part of this project will be freely distributed on both university and external websites.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

人类语言被认为允许使用有限的规则集（称为语法）来表达无限的可能含义，这些语法为单词赋予含义，并将单词和短语的含义组合成更大的短语和从句。对目标和世界行为的精确描述——语言学家对这种精确交流所涉及的语言逻辑结构了解很多——但语言学的核心开放问题之一是人类如何获得这些语法的计算模型。从记录或转录的话语中学习的数据可以提供证据，证明这种学习可以由没有明显先天生物学偏见的儿童完成，并且这些模型还可以提供用于分析和记录濒危语言的自动化工具，包括许多现有的美国本土统计和神经语言。语法语言学习方法可以从文本语料库中的句子中归纳出语法，这些语法可以预测语言学家注释的大约一半的短语和从句，但是这种性能水平远不及人类语言学习者的准确性，并且尝试使用图像来支持这种学习；和视频数据并没有显着提高归纳准确性。拟议的工作将从大型商用神经语言模型中提取有关逻辑谓词的统计数据作为人类世界知识的替代品，以提高语法归纳的准确性。拟议的工作将开发第一个广泛覆盖的语义语法归纳模型，通过从大型预训练神经语言模型中提取世界知识，将其集成到获取过程中。大型语言模型中隐含的世界知识将使用谓词共现统计数据被提取到矩阵中。论证特定的生成的谓词共现统计不会区分主动句和被动句、主题句和非主题句、陈述句和主助倒装句。该模型将用于评估有关统计的主张。拟议的工作还将继续开发用于评估这些结构模型的资源，作为该项目的一部分收集的模型和语料库将在大学和外部网站上免费分发。该奖项反映了 NSF 的法定使命和目标。通过使用基金会的智力价值和更广泛的影响审查标准进行评估，该项目被认为值得支持。