Collaborative Research: IIS: III: MEDIUM: Learning Protein-ish: Foundational Insight on Protein Language Models for Better Understanding, Democratized Access, and Discovery

协作研究：IIS：III：中等：学习蛋白质：对蛋白质语言模型的基础洞察，以更好地理解、民主化访问和发现

• 批准号: 2310113
• 负责人: Amarda Shehu
• 金额: $ 59.99万
• 依托单位: George Mason University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-08-01 至 2026-07-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2310113&HistoricalAwards=false
• 关键词: Collaborative; Research; IIS; III; MEDIUM

Large language models are massive neural networks that learn rich contextual representations of words and use such representations to address a variety of tasks in natural language processing (NLP). These models are a prominent example of generative artificial intelligence and are emerging as promising approaches for distilling and organizing the content of massive biological databases and for predicting a wide range of molecular bio-properties. Yet, we know surprisingly little about what these models capture in their learned representations, why they perform well on some tasks and not on others, and how they can produce deep insight into the relationships describing the biological space. If progress in NLP is any indication, the current trend of improving the performance of language models by drastically increasing the number of their trainable parameters is unsustainable both for our carbon footprint and for ensuring equity/accessibility of research and scholarship in the academic setting. This project advances algorithmic research at the intersection of information integration and informatics using principled protein language models (PLMs) as computational vehicles for deeper insight into the structural, functional, and evolutionary organization across protein space at varying levels of detail and scale. It also aims to do so in a way that is resource-aware, sustainable, and accessible to all researchers. The research activities are organized in three thrusts: (1) encoding prior biological knowledge in PLMs for joint and resource-aware learning in composite spaces, (2) revealing fundamental properties and organizing the learned representation space to inform and connect what is captured with properties of interest, and (3) enabling PLMs to capture diverse contexts for deeper exploration of the structural, functional, and evolutionary organization across protein space. This interdisciplinary approach contributes to the fields of machine learning, bioinformatics, and molecular biology and provides opportunities at the interface of these disciplines for training under-represented students of all levels. The investigators are determined to bridge communities and disciplines, and they have planned activities to build and galvanize a trans-disciplinary community to further advance their research.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

大型语言模型是大型神经网络，可以学习丰富的单词上下文表示，并使用此类表示来解决自然语言处理 (NLP) 中的各种任务。这些模型是生成人工智能的一个突出例子，并且正在成为提炼和组织大规模生物数据库内容以及预测各种分子生物特性的有前途的方法。然而，令人惊讶的是，我们对这些模型在其学习表征中捕捉到的内容、为什么它们在某些任务上表现良好而在其他任务上表现不佳，以及它们如何能够深入了解描述生物空间的关系知之甚少。如果 NLP 的进展有任何迹象的话，那么当前通过大幅增加可训练参数的数量来提高语言模型性能的趋势对于我们的碳足迹以及确保学术环境中研究和学术的公平/可及性来说都是不可持续的。该项目使用原则性蛋白质语言模型（PLM）作为计算工具，推进信息集成和信息学交叉领域的算法研究，以更深入地了解蛋白质空间中不同细节和规模的结构、功能和进化组织。它还旨在以一种资源意识强、可持续且可供所有研究人员使用的方式来实现这一目标。研究活动分为三个重点：(1) 在 PLM 中编码先验生物知识，以便在复合空间中进行联合和资源感知学习，(2) 揭示基本属性并组织学习的表示空间，以告知和连接所捕获的内容与属性(3) 使 PLM 能够捕获不同的背景，以便更深入地探索整个蛋白质空间的结构、功能和进化组织。这种跨学科方法有助于机器学习、生物信息学和分子生物学领域的发展，并为这些学科的交叉点提供机会，以培训各个级别的代表性不足的学生。研究人员决心在社区和学科之间架起桥梁，并计划开展活动来建立和激励跨学科社区，以进一步推进他们的研究。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值进行评估，被认为值得支持以及更广泛的影响审查标准。