RI: Small: Extracting and Representing Commonsense Knowledge Using Language Models
RI:小:使用语言模型提取和表示常识知识
基本信息
- 批准号:2006851
- 负责人:
- 金额:$ 47万
- 依托单位:
- 依托单位国家:美国
- 项目类别:Standard Grant
- 财政年份:2020
- 资助国家:美国
- 起止时间:2020-10-01 至 2024-09-30
- 项目状态:已结题
- 来源:
- 关键词:
项目摘要
As computers advance from serving as our tools to becoming our helpers and collaborators, they must be capable of commonsense reasoning--for example, knowing that a person needs to use their hand to open a door (and that this might be a problem if they are carrying groceries). This kind of commonsense reasoning is a longstanding, elusive goal of artificial intelligence, but is becoming within reach today due to the availability of vast amounts of data and more powerful computational models for learning from those data. This project is aimed at a key step in enabling commonsense reasoning by machines: the automatic acquisition of common sense knowledge. The project’s approach builds upon recent breakthroughs in language models that learn by reading large amounts of text, and combines these in novel ways with explicit commonsense knowledge gathered from humans. The project probes new, scalable methods for humans to impart their commonsense knowledge to the system, by using existing dictionaries and encyclopedias, building curricula that help machines build to commonsense mastery step by step, and by directly enforcing key logical constraints (for example, that if one item is bigger than another, then the second item must be smaller than the first). Success in this project could help power new virtual assistants, medical diagnosis and treatment systems, improved search engines, and other important applications of AI. The work also aims to enable the development of better language models themselves---improving current commercial technologies such as speech recognition and machine translation, and ultimately helping to power the next generation of computer systems capable of communicating with people more naturally using language. Along the way, the project will help train the next generation of students about these approaches and technologies, via education and outreach activities.The technical strategy used in the project involves learning unsupervised neural language models (LMs) to capture textual distributions, and then extracting common sense knowledge from those models. This approach is challenging because common sense knowledge is multifarious and massive, and yet is not often explicitly stated in text. The project aims to overcome this challenge using several methods for scalably incorporating human input in concert with neural language models. First, the project studies how to use explicit lexical knowledge found in dictionaries to improve LMs, extending prior work in modeling the definitions of terms with neural LMs. Next, the project is investigating a “scaffold” of semantic tasks (a task curriculum of increasing complexity) incrementally constructing models for each task in turn in a way that aims to improve the learning of each subsequent task. Third, the project is developing methods for encoding commonsense logical constraints within neural language models. Lastly, because time and energy cost is a potential barrier to the application of the proposed techniques, the project is also studying how to make its approaches efficient. In particular, the project is investigating ways to scale-up LMs to larger corpora while reducing the significant computational and energy cost in LM training, by learning how to automatically identify text that will be more informative for training.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
随着计算机从作为我们的工具成为我们的助手和合作者的工具时,他们必须能够打交道推理 - 例如,知道一个人需要用手打开门(如果他们携带杂货,这可能是一个问题)。这种常识性推理是一个长期以来的人工智能目标,但是由于大量数据和更强大的计算模型,可以从这些数据中学习。该项目是针对通过机器实现常识推理的关键步骤:自动获取常识知识。该项目的方法是基于最近通过阅读大量文本学习的语言模型的突破,并以新颖的方式与从人类收集的明确常识性知识相结合。该项目通过使用现有词典和百科全书,构建帮助机器逐步构建的课程,并通过直接执行关键的逻辑约束来构建新的,可扩展的方法,以使人类将其常识性知识传授给系统,并建立帮助机器构建的课程(例如,如果一个项目比另一个项目更大,那么一个项目必须小于第一个项目)。该项目的成功可以帮助推动新的虚拟助手,医学诊断和治疗系统,改进的搜索引擎以及AI的其他重要应用。这项工作还旨在使更好的语言模型本身的开发 - 改进当前的商业技术,例如语音识别和机器翻译,并最终有助于为下一代的计算机系统提供动力,能够使用语言更自然地与人们进行交流。在此过程中,该项目将通过教育和外展活动帮助培训下一代学生有关这些方法和技术的培训。该项目中使用的技术策略涉及学习无监督的中性语言模型(LMS),以捕获文本分布,然后从这些模型中提取常识知识。这种方法是挑战,因为常识知识是多种多样和庞大的,但经常在文本中明确说明。该项目旨在使用几种与神经语言模型共同增加人类投入的方法来克服这一挑战。首先,该项目研究如何使用词典中发现的明确词汇知识来改善LMS,从而扩展了用神经LMS对术语进行建模的先前工作。接下来,该项目正在调查语义任务的“脚手架”(增加复杂性的任务课程),以依次为每个任务的模型逐步构造模型,旨在改善每个后续任务的学习。第三,该项目正在开发用于编码神经语言模型中常识性逻辑约束的方法。最后,由于时间和能源成本是所提出技术应用的潜在障碍,因此该项目还在研究如何使其方法有效。特别是,该项目正在研究通过学习如何自动识别如何自动识别培训内容的文本,同时降低LM培训中的大量计算和能源成本的方法,同时降低LM培训中的重大计算和能源成本。该奖项反映了NSF的法定任务,并认为通过使用该基金会的知识分子和更广泛的影响来评估NSF的法定任务,并被认为是值得的。
项目成果
期刊论文数量(4)
专著数量(0)
科研奖励数量(0)
会议论文数量(0)
专利数量(0)
Who’s on First?: Probing the Learning and Representation Capabilities of Language Models on Deterministic Closed Domains
- DOI:10.18653/v1/2021.conll-1.16
- 发表时间:2021
- 期刊:
- 影响因子:0
- 作者:David Demeter;Doug Downey
- 通讯作者:David Demeter;Doug Downey
CODE: Compiler-based Neuron-aware Ensemble training
- DOI:
- 发表时间:2021
- 期刊:
- 影响因子:0
- 作者:E. Trainiti;Thanapon Noraset;David Demeter;Doug Downey;Simone Campanoni
- 通讯作者:E. Trainiti;Thanapon Noraset;David Demeter;Doug Downey;Simone Campanoni
Learning to Perform Complex Tasks through Compositional Fine-Tuning of Language Models
通过语言模型的组合微调学习执行复杂任务
- DOI:10.18653/v1/2022.findings-emnlp.121
- 发表时间:2022
- 期刊:
- 影响因子:0
- 作者:Bursztyn, Victor;Demeter, David;Downey, Doug;Birnbaum, Larry
- 通讯作者:Birnbaum, Larry
“It doesn’t look good for a date”: Transforming Critiques into Preferences for Conversational Recommendation Systems
“约会看起来不太好”:将批评转化为对话推荐系统的偏好
- DOI:10.18653/v1/2021.emnlp-main.145
- 发表时间:2021
- 期刊:
- 影响因子:0
- 作者:Bursztyn, Victor;Healey, Jennifer;Lipka, Nedim;Koh, Eunyee;Downey, Doug;Birnbaum, Larry
- 通讯作者:Birnbaum, Larry
{{
item.title }}
{{ item.translation_title }}
- DOI:
{{ item.doi }} - 发表时间:
{{ item.publish_year }} - 期刊:
- 影响因子:{{ item.factor }}
- 作者:
{{ item.authors }} - 通讯作者:
{{ item.author }}
数据更新时间:{{ journalArticles.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ monograph.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ sciAawards.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ conferencePapers.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ patent.updateTime }}
Douglas Downey其他文献
Douglas Downey的其他文献
{{
item.title }}
{{ item.translation_title }}
- DOI:
{{ item.doi }} - 发表时间:
{{ item.publish_year }} - 期刊:
- 影响因子:{{ item.factor }}
- 作者:
{{ item.authors }} - 通讯作者:
{{ item.author }}
{{ truncateString('Douglas Downey', 18)}}的其他基金
CAREER: Web Information Extraction: Integration and Scaling
职业:Web 信息提取:集成和扩展
- 批准号:
1351029 - 财政年份:2014
- 资助金额:
$ 47万 - 项目类别:
Continuing Grant
RI: Medium: Collaborative Research: Learning Representations of Language for Domain Adaptation
RI:媒介:协作研究:学习领域适应的语言表示
- 批准号:
1065270 - 财政年份:2011
- 资助金额:
$ 47万 - 项目类别:
Continuing Grant
III: Small: Active Learning of Language Models for Information Extraction
三:小:用于信息提取的语言模型的主动学习
- 批准号:
1016754 - 财政年份:2010
- 资助金额:
$ 47万 - 项目类别:
Standard Grant
相似国自然基金
基于Rab8A介导的小胶质细胞铁稳态及神经炎症调控探讨灵芝提取物改善LRRK2-G2019S突变介导的帕金森病的机制
- 批准号:82305015
- 批准年份:2023
- 资助金额:30 万元
- 项目类别:青年科学基金项目
小域上保密信息提取的方案构造问题研究
- 批准号:12201362
- 批准年份:2022
- 资助金额:30.00 万元
- 项目类别:青年科学基金项目
卫星重力提取中-小尺度陆地水变化的先验信息约束与联合反演研究
- 批准号:42274110
- 批准年份:2022
- 资助金额:56 万元
- 项目类别:面上项目
小域上保密信息提取的方案构造问题研究
- 批准号:
- 批准年份:2022
- 资助金额:30 万元
- 项目类别:青年科学基金项目
卫星重力提取中-小尺度陆地水变化的先验信息约束与联合反演研究
- 批准号:
- 批准年份:2022
- 资助金额:56 万元
- 项目类别:面上项目
相似海外基金
RI: Small: Extracting Knowledge from Language Models for Decision Making
RI:小型:从语言模型中提取知识以进行决策
- 批准号:
2246811 - 财政年份:2023
- 资助金额:
$ 47万 - 项目类别:
Standard Grant
III: Small: Accessible and Interpretable Machine Reading Methods for Extracting Structured Information from Text
III:小:从文本中提取结构化信息的可访问且可解释的机器阅读方法
- 批准号:
2006583 - 财政年份:2020
- 资助金额:
$ 47万 - 项目类别:
Continuing Grant
RI: Small: Collaborative Research: Extracting Dynamics from Limited Data for Modeling and Control of Unmanned Autonomous Systems
RI:小型:协作研究:从有限数据中提取动力学,用于无人自主系统的建模和控制
- 批准号:
1910308 - 财政年份:2019
- 资助金额:
$ 47万 - 项目类别:
Standard Grant
RI: Small: Collaborative Research: Extracting Dynamics from Limited Data for Modeling and Control of Unmanned Autonomous Systems
RI:小型:协作研究:从有限数据中提取动力学,用于无人自主系统的建模和控制
- 批准号:
1910087 - 财政年份:2019
- 资助金额:
$ 47万 - 项目类别:
Standard Grant
CHS: Small: Extracting affect and interaction information from primary care visits to support patient-provider interactions
CHS:小型:从初级保健就诊中提取情感和互动信息,以支持患者与提供者之间的互动
- 批准号:
1816010 - 财政年份:2018
- 资助金额:
$ 47万 - 项目类别:
Standard Grant