EAGER: Integrating Pathological Image and Biomedical Text Data for Clinical Outcome Prediction

EAGER：整合病理图像和生物医学文本数据进行临床结果预测

• 批准号: 2412195
• 负责人: Junzhou Huang
• 金额: $ 20万
• 依托单位: University of Texas at Arlington
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2024
• 资助国家: 美国
• 起止时间: 2024-03-15 至 2026-02-28
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2412195&HistoricalAwards=false
• 关键词: EAGER; Integrating; Pathological; Image; Biomedical

The accurate prediction of clinical outcomes is a critical aspect of personalized medicine, offering vital information that can shape patient treatment plans and ultimately affect patient prognosis. The current pathological grading/classification system requires extensive information processing by a human brain to interpret highly complex data resources. Histopathology, as the cornerstone of disease diagnosis, has advanced significantly with technological innovations allowing for the capture of images at greater speed and resolution. However, most current histopathological image analysis methods often overlook the complex hierarchical structures of tissues. Understanding the intricate interactions among various cell types, which form the cellular components and, in turn, tissue architectures, is crucial for insights into biology and disease status. Analyzing pathological images is crucial, but effectively integrating associated biomedical text data, such as pathological reports, preliminary diagnosis reports, and clinical notes, poses additional challenges. This variety of text data is defined as medical captions, akin to extended image captions, which provide necessary context but also introduce further complexity in diagnostics. Enhanced computational methods that simultaneously leverage pathological slides and their captions could revolutionize the accuracy and efficiency of predicting clinical outcomes.The goal of this project is to develop novel pathological image-text analysis tools for clinical outcome prediction. The project will focus on 1) developing algorithms for pathological image analysis, which include auto-prompting fine-tuning framework for subtype cell segmentation, cell-level graph learning, patch-level graph learning, and intelligent integration of cell-level graph and patch-level graph for clinical outcome prediction; 2) fine-tuning large language models using biomedical text data to obtain improved text embeddings, which include the development of algorithms for biomedical text data analysis, incorporating fine-tuning of deep pre-trained models for precise biomedical text data representation; 3) Integrating pathological image data with biomedical text data for clinical outcome prediction, which include novel algorithms for the intelligent integration of multi-modal data and cross-modal learning models to generate biomedical text data representation from the histopathological images of the same patient. The successful realization of these aims promises to provide healthcare professionals with powerful tools to enhance the decision-making process, personalize treatment plans, and improve overall patient outcomes. Additionally, the proposed study stands to offer broader insights into the integration of multi-modal medical data, setting a new standard for how medical informatics can be leveraged in the era of big data and precision medicine. The multidisciplinary nature of this project also provides unique opportunities for integrating its components into existing curricula, as well as inspiring scientific interests in K-12 students and underrepresented students. The results of this project will be disseminated in the form of peer-reviewed publications, open-source software, tutorials, seminars, and workshops.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

对临床结果的准确预测是个性化医学的关键方面，它提供了可以塑造患者治疗计划并最终影响患者预后的重要信息。当前的病理分级/分类系统需要人脑的广泛信息处理来解释高度复杂的数据资源。作为疾病诊断的基石，组织病理学已随着技术创新的显着发展，允许以更高的速度和分辨率捕获图像。但是，大多数当前的组织病理学图像分析方法通常忽略组织的复杂分层结构。了解形成细胞成分的各种细胞类型之间的复杂相互作用，进而对生物学和疾病状态的见解至关重要。分析病理图像至关重要，但有效地整合了相关的生物医学文本数据，例如病理报告，初步诊断报告和临床注释，提出了其他挑战。这些多种文本数据被定义为医学字幕，类似于扩展的图像标题，这些字幕提供了必要的上下文，但也引入了诊断的进一步复杂性。增强的计算方法，同时利用病理幻灯片及其标题可以彻底改变预测临床结果的准确性和效率。该项目的目的是开发用于临床结果预测的新型病理图像文本分析工具。该项目将重点关注1）开发用于病理图像分析的算法，其中包括用于亚型细胞分割的自动促进微调框架，细胞级图形学习，补丁级图形学习以及细胞级图形和贴平级图的智能集成以进行临床结果预测； 2）使用生物医学文本数据对大型语言模型进行微调模型，以获得改进的文本嵌入，其中包括开发用于生物医学文本数据分析的算法，并结合了对精确生物医学文本数据表示的深训练模型的微调； 3）将病理图像数据与生物医学文本数据相结合，以进行临床结果预测，其中包括用于智能整合多模式数据和跨模式学习模型的新算法，以从同一患者的组织病理学图像中生成生物医学文本数据表示。这些目标的成功实现有望为医疗保健专业人员提供强大的工具，以增强决策过程，个性化治疗计划并改善整体患者的结果。此外，拟议的研究将为多模式医学数据的整合提供更广泛的见解，为在大数据和精密医学时代如何利用医学信息学设定了新的标准。该项目的多学科性质还提供了将其组成部分整合到现有课程中的独特机会，并激发了K-12学生和代表性不足的学生的科学兴趣。该项目的结果将以经过同行评审的出版物，开源软件，教程，研讨会和讲习班的形式进行传播。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的知识分子优点和更广泛的审查标准通过评估来通过评估来支持的。