Exploring Better Distributed Representation and Composition Models for Semantics

探索更好的分布式语义表示和组合模型

• 批准号: RGPIN-2018-06415
• 负责人: Zhu, Xiaodan
• 金额: $ 4.37万
• 依托单位: Queen's University
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2022
• 资助国家: 加拿大
• 起止时间: 2022-01-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=752328
• 关键词: Exploring; Better; Distributed; Representation; Composition

Computers understand very little of the meaning of human language. However, language data permeates almost all aspects of our daily life, e.g., news, emails, social media, telephone conversations, medical records, books, just to name a few. Search engines like Google are only scratching the surface of human language, and yet the impact on society and the economy is already immense. Developing better semantic modelling technologies will have extensive impact on real-life applications. In general, machine understanding of natural language with human-level performance remains one of the grand challenges of artificial intelligence (AI), where modelling meaning is a major obstacle to achieving the goal. The long-term objective of this program is to devise state-of-the-art computational models that learn better representations for text. Specifically, we focus on distributed representations, including those learned with neural network models, in which text is represented as real-valued vectors. If properly learned, such representations have been shown to be very effective, resulting in cutting-edge performance on a wide range of natural language processing (NLP) problems. Building on our past contributions, this research program comprises three specific sets of short-term objectives for advancing distributed representations and neural network models for semantic representation. The first set aims to contribute novel algorithms for semantic composition. The principle of compositionality, positing that the meaning of a whole is a (complicated) function of the meaning of the parts, is a fundamental approach to learning representation for natural language. Compositionality is in general regarded by many as a basic ingredient of human intelligence. Based on our recent progress, this proposal aims to explore better neural-network-based semantic composition by further investigating external knowledge. In this set of objectives, we will also further explore non-compositionality, a basic phenomenon in language, in neural composition models. In the second set of objectives, we will delve into several specific aspects of semantics to deepen our understanding of distributed representation and neural-network-based modelling. While distributed representation has been shown to be very effective for modelling similarity and relatedness, its effectiveness for representing other key aspects of semantics, such as contrasting meaning and entailment, requires more investigation. The third set of objectives attempt to learn better distributed representation by further exploring syntactic structures. Language is rich in structures. It still remains a widely open question as to how to effectively consider such structures with the demonstrated modelling ability of distributed representation and neural networks.

计算机对人类语言的含义几乎没有理解。但是，语言数据几乎渗透到我们日常生活的所有方面，例如新闻，电子邮件，社交媒体，电话对话，医疗记录，书籍，仅举几例。诸如Google之类的搜索引擎只是刮擦人类语言的表面，但对社会和经济的影响已经很大。开发更好的语义建模技术将对现实生活中的应用产生广泛的影响。一般而言，机器对自然语言具有人类水平表现的理解仍然是人工智能（AI）的巨大挑战之一，在该挑战中，建模含义是实现目标的主要障碍。 该计划的长期目标是设计最先进的计算模型，以了解文本的更好表示。具体而言，我们专注于分布式表示，包括具有神经网络模型的人，其中文本表示为实价矢量。如果有足够的学习，这些表示形式已被证明非常有效，从而在广泛的自然语言处理（NLP）问题上产生了尖端的表现。 在我们过去的贡献的基础上，该研究计划包括三组短期目标，用于推进分布式表示形式和神经网络模型以进行语义表示。第一组旨在为语义组成贡献新颖的算法。构图的原则认为，整体的含义是部分的含义的（复杂）功能，是一种学习自然语言表示表示的基本方法。总体上，组成性被许多人认为是人类智力的基本要素。根据我们最近的进步，该提案旨在通过进一步研究外部知识来探索更好的基于神经网络的语义组成。在这组目标中，我们还将进一步探索神经组成模型中的非复合性，这是一种语言的基本现象。在第二组目标中，我们将研究语义的几个特定方面，以加深我们对分布式表示和基于神经网络的建模的理解。尽管已证明分布式表示对于建模相似性和相关性非常有效，但其代表语义的其他关键方面的有效性（例如对比含义和影响）需要进行更多的研究。第三组目标试图通过进一步探索句法结构来学习更好的分布式表示。语言富含结构。关于如何具有分布式表示和神经网络的建模能力有效地考虑这样的结构，这仍然是一个广泛的问题。