Exploring Better Distributed Representation and Composition Models for Semantics

探索更好的分布式语义表示和组合模型

• 批准号: RGPIN-2018-06415
• 负责人: Zhu, Xiaodan
• 金额: $ 4.37万
• 依托单位: Queen's University
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2022
• 资助国家: 加拿大
• 起止时间: 2022-01-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=752328
• 关键词: Exploring; Better; Distributed; Representation; Composition

Computers understand very little of the meaning of human language. However, language data permeates almost all aspects of our daily life, e.g., news, emails, social media, telephone conversations, medical records, books, just to name a few. Search engines like Google are only scratching the surface of human language, and yet the impact on society and the economy is already immense. Developing better semantic modelling technologies will have extensive impact on real-life applications. In general, machine understanding of natural language with human-level performance remains one of the grand challenges of artificial intelligence (AI), where modelling meaning is a major obstacle to achieving the goal. The long-term objective of this program is to devise state-of-the-art computational models that learn better representations for text. Specifically, we focus on distributed representations, including those learned with neural network models, in which text is represented as real-valued vectors. If properly learned, such representations have been shown to be very effective, resulting in cutting-edge performance on a wide range of natural language processing (NLP) problems. Building on our past contributions, this research program comprises three specific sets of short-term objectives for advancing distributed representations and neural network models for semantic representation. The first set aims to contribute novel algorithms for semantic composition. The principle of compositionality, positing that the meaning of a whole is a (complicated) function of the meaning of the parts, is a fundamental approach to learning representation for natural language. Compositionality is in general regarded by many as a basic ingredient of human intelligence. Based on our recent progress, this proposal aims to explore better neural-network-based semantic composition by further investigating external knowledge. In this set of objectives, we will also further explore non-compositionality, a basic phenomenon in language, in neural composition models. In the second set of objectives, we will delve into several specific aspects of semantics to deepen our understanding of distributed representation and neural-network-based modelling. While distributed representation has been shown to be very effective for modelling similarity and relatedness, its effectiveness for representing other key aspects of semantics, such as contrasting meaning and entailment, requires more investigation. The third set of objectives attempt to learn better distributed representation by further exploring syntactic structures. Language is rich in structures. It still remains a widely open question as to how to effectively consider such structures with the demonstrated modelling ability of distributed representation and neural networks.

计算机对人类语言的含义知之甚少。然而，语言数据几乎渗透到我们日常生活的各个方面，例如新闻、电子邮件、社交媒体、电话交谈、医疗记录、书籍等等。像谷歌这样的搜索引擎只触及了人类语言的表面，但对社会和经济的影响已经是巨大的。开发更好的语义建模技术将对现实生活中的应用产生广泛的影响。总的来说，机器对自然语言的理解与人类水平的表现仍然是人工智能（AI）的巨大挑战之一，其中建模意义是实现这一目标的主要障碍。 该计划的长期目标是设计最先进的计算模型，以学习更好的文本表示。具体来说，我们专注于分布式表示，包括通过神经网络模型学习的分布式表示，其中文本表示为实值向量。如果学习得当，这种表示形式已被证明非常有效，可以在各种自然语言处理（NLP）问题上取得尖端性能。 基于我们过去的贡献，该研究计划包括三组具体的短期目标，用于推进分布式表示和语义表示的神经网络模型。第一组旨在为语义组合贡献新颖的算法。组合性原则假设整体的意义是部分意义的（复杂）函数，是学习自然语言表示的基本方法。许多人普遍认为组合性是人类智力的基本组成部分。基于我们最近的进展，该提案旨在通过进一步研究外部知识来探索更好的基于神经网络的语义组合。在这组目标中，我们还将进一步探索神经组合模型中的非组合性（语言的基本现象）。在第二组目标中，我们将深入研究语义的几个具体方面，以加深我们对分布式表示和基于神经网络的建模的理解。虽然分布式表示已被证明对于建模相似性和相关性非常有效，但其表示语义其他关键方面（例如对比意义和蕴涵）的有效性还需要更多研究。第三组目标试图通过进一步探索句法结构来学习更好的分布式表示。语言具有丰富的结构。如何利用分布式表示和神经网络已证明的建模能力来有效地考虑这种结构，仍然是一个广泛悬而未决的问题。