Navigating Chemical Space with Natural Language Processing and Deep Learning

利用自然语言处理和深度学习驾驭化学空间

• 批准号: EP/Y004167/1
• 负责人: Jiayun Pang
• 金额: $ 11.41万
• 依托单位: University of Greenwich
• 依托单位国家: 英国
• 项目类别: Research Grant
• 财政年份: 2024
• 资助国家: 英国
• 起止时间: 2024 至 无数据
• 项目状态: 未结题
• 来源: https://gtr.ukri.org/projects?ref=EP%2FY004167%2F1
• 关键词: Navigating; Chemical; Space; Natural; Language

Natural language processing (NLP) lies at the intersection between linguistics and computer science which aims to process and analyse human language, typically provided as written text. NLP is now strongly focused on the use of machine learning for challenging tasks with some revolutionary algorithms having been developed in the last few years. They now underpin a wide range of real-life applications, such as ChatGPT, virtual assistants and automatic text completion when we write emails. Innovative research ideas often come from integrating techniques and concepts across disciplines. For this discipline-hopping grant, we would like to explore how Transformer models, a ground-breaking deep learning algorithm developed by Google in 2017 which fuels majority of the current cutting-edge research in NLP, can be adapted to solve research challenges in chemistry. Chemical structures are usually three dimensional. However, they are also often converted into sequences, called SMILES. SMILES has a simple vocabulary of chemical elements and bond symbols and a few grammatical rules of how the chemical elements are positioned. Owing to this direct analogy to text sequences, through SMILES it is possible to use NLP algorithms to analyse chemical structures in a similar fashion as they are used to analyse text. For the proposed research, Dr Pang, a chemist will work with Dr Vulic, an NLP and machine learning expert in order to get up to speed with the latest developments in the field of NLP and to examine their further applicability in her domain of expertise. We will explore and utilise a concept which is now pervasive in machine learning and NLP, termed transfer learning, which 1) pretrains large general-purpose models, and 2) fine-tunes (i.e., specialises) those general models for specific tasks and applications, where labelled data are expensive to create (as they require expert knowledge and complex annotation protocols) and thus inherently scarce. Specifically, we will pretrain Transformer models to learn a latent representation of the chemical space defined by tens of millions of SMILES. This learned latent representation can then be used to predict molecular properties for a given chemical structure during fine-tuning. The advantage of this type of approach is that the resulting machine learning models rely less on the so-called labelled data (molecules with experimentally determined properties), which are time-consuming or even impossible to generate in chemistry considering the associated cost and experimental challenges. We will aim to make the Transformer models more computationally efficient and accurate using two latest machine learning techniques, termed sentence encoding and contrastive learning. We hope that this new molecular representation can complement existing molecular representation methods and provide an alternative approach to evaluate molecular structures against their properties, which underpins many research and development tasks in the chemical and pharmaceutical industries.

自然语言处理（NLP）位于语言学和计算机科学的交叉点，旨在处理和分析通常以书面文本形式提供的人类语言。 NLP 现在重点关注使用机器学习来完成具有挑战性的任务，过去几年开发了一些革命性的算法。它们现在支撑着广泛的现实生活应用程序，例如 ChatGPT、虚拟助理和我们编写电子邮件时的自动文本完成。创新的研究想法往往来自跨学科的技术和概念的整合。对于这笔跨学科资助，我们希望探索 Transformer 模型（谷歌于 2017 年开发的一种突破性深度学习算法，为当前 NLP 领域的大部分前沿研究提供动力）如何适用于解决化学领域的研究挑战。化学结构通常是三维的。然而，它们也经常被转换成序列，称为 SMILES。 SMILES 有一个简单的化学元素和键符号词汇表以及一些化学元素如何定位的语法规则。由于这种与文本序列的直接类比，通过 SMILES，可以使用 NLP 算法以与分析文本类似的方式来分析化学结构。对于拟议的研究，化学家 Pang 博士将与 NLP 和机器学习专家 Vulic 博士合作，以便跟上 NLP 领域的最新发展，并检验它们在她的专业领域的进一步适用性。我们将探索和利用一个现在在机器学习和 NLP 中普遍存在的概念，称为迁移学习，它 1）预训练大型通用模型，2）针对特定任务和应用程序微调（即专门化）这些通用模型，其中标记数据的创建成本很高（因为它们需要专业知识和复杂的注释协议），因此本质上是稀缺的。具体来说，我们将预训练 Transformer 模型，以学习由数千万个 SMILES 定义的化学空间的潜在表示。然后，这种学习到的潜在表示可用于在微调过程中预测给定化学结构的分子特性。这种方法的优点是，所得的机器学习模型较少依赖所谓的标记数据（具有实验确定属性的分子），考虑到相关的成本和实验挑战，这些数据在化学中非常耗时，甚至不可能生成。我们的目标是使用两种最新的机器学习技术（称为句子编码和对比学习）使 Transformer 模型在计算上更加高效和准确。我们希望这种新的分子表示可以补充现有的分子表示方法，并提供一种根据分子特性评估分子结构的替代方法，这为化学和制药行业的许多研究和开发任务奠定了基础。