Collaborative Research: IIS: III: MEDIUM: Learning Protein-ish: Foundational Insight on Protein Language Models for Better Understanding, Democratized Access, and Discovery

协作研究：IIS：III：中等：学习蛋白质：对蛋白质语言模型的基础洞察，以更好地理解、民主化访问和发现

• 批准号: 2310113
• 负责人: Amarda Shehu
• 金额: $ 59.99万
• 依托单位: George Mason University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-08-01 至 2026-07-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2310113&HistoricalAwards=false
• 关键词: Collaborative; Research; IIS; III; MEDIUM

Large language models are massive neural networks that learn rich contextual representations of words and use such representations to address a variety of tasks in natural language processing (NLP). These models are a prominent example of generative artificial intelligence and are emerging as promising approaches for distilling and organizing the content of massive biological databases and for predicting a wide range of molecular bio-properties. Yet, we know surprisingly little about what these models capture in their learned representations, why they perform well on some tasks and not on others, and how they can produce deep insight into the relationships describing the biological space. If progress in NLP is any indication, the current trend of improving the performance of language models by drastically increasing the number of their trainable parameters is unsustainable both for our carbon footprint and for ensuring equity/accessibility of research and scholarship in the academic setting. This project advances algorithmic research at the intersection of information integration and informatics using principled protein language models (PLMs) as computational vehicles for deeper insight into the structural, functional, and evolutionary organization across protein space at varying levels of detail and scale. It also aims to do so in a way that is resource-aware, sustainable, and accessible to all researchers. The research activities are organized in three thrusts: (1) encoding prior biological knowledge in PLMs for joint and resource-aware learning in composite spaces, (2) revealing fundamental properties and organizing the learned representation space to inform and connect what is captured with properties of interest, and (3) enabling PLMs to capture diverse contexts for deeper exploration of the structural, functional, and evolutionary organization across protein space. This interdisciplinary approach contributes to the fields of machine learning, bioinformatics, and molecular biology and provides opportunities at the interface of these disciplines for training under-represented students of all levels. The investigators are determined to bridge communities and disciplines, and they have planned activities to build and galvanize a trans-disciplinary community to further advance their research.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

大型语言模型是庞大的神经网络，可以学习文字的丰富上下文表示，并使用此类表示来解决自然语言处理（NLP）的各种任务。这些模型是生成人工智能的重要例子，并且正在成为蒸馏和组织大量生物学数据库含量并预测广泛的分子生物培训的有前途的方法。但是，我们对这些模型在他们所学的表示中所捕获的内容，为什么在某些任务而不是在其他任务上表现良好，以及他们如何对描述生物学空间的关系产生深刻的见解，这一点却一无所知。如果NLP的进展是任何迹象，那么通过大幅度增加其可训练参数的数量来改善语言模型的表现的当前趋势对于我们的碳足迹而言是不可持续的，并且可以在学术环境中确保研究和奖学金的公平性/可及性。该项目使用原则性的蛋白质语言模型（PLM）作为计算工具，在信息整合和信息学的交集中推进算法研究，以更深入地了解跨蛋白质空间的结构，功能和进化组织，这些蛋白质在各种详细水平和规模的水平上。它还旨在以所有研究人员的资源感知，可持续和访问的方式这样做。研究活动有三个力量组织：（1）编码PLM中的先前的生物学知识，用于复合空间中的联合和资源感知学习，（2）揭示基本属性并组织学习的代表空间，以将捕获的属性告知和连接与兴趣的属性，（3）使PLMS捕捉到具有更深层探索的多样性的构造的概念，并构成了构造的组织，并进行了努力，并进行了效率，并进行了效率，并浏览了效率，并进行了效率。这种跨学科的方法有助于机器学习，生物信息学和分子生物学的领域，并在这些学科的界面上提供了机会，以培训各个层次的代表性不足的学生。调查人员决心弥合社区和学科，他们计划进行跨学科社区建立和激发其进一步促进其研究的活动。该奖项反映了NSF的法定使命，并被认为是值得通过基金会的知识分子优点和更广泛的审查标准通过评估来进行评估的。