Collaborative Research:SCH:Bimodal Interpretable Multi-Instance Medical-Image Classification

合作研究：SCH：双峰可解释多实例医学图像分类

• 批准号: 2306572
• 负责人: Tianbao Yang
• 金额: $ 85.5万
• 依托单位: Texas A&M Engineering Experiment Station
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-09-01 至 2027-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2306572&HistoricalAwards=false
• 关键词: Collaborative; Research; SCH; Bimodal; Interpretable

This project focuses on creating a smarter artificial-intelligence (AI) system to better understand and analyze complex medical images, such as those from multiple scans of a patient. Traditional methods have had some success but face challenges in dealing with rare diseases and in providing explanations that doctors and patients can easily understand. This project aims to develop a modern AI approach that overcomes these limitations by leveraging a vast collection of medical images and doctors' notes, regardless of the specific health conditions to which they pertain. The research team will tackle various challenges to make the AI system more scalable, interpretable, and robust. This innovative project will deliver trustworthy AI-driven diagnostic tools to medical workers, expediting the diagnostic process for complex medical images. The impact of this project will be felt broadly in AI research and beyond, as its foundational research is likely to have impact in various applications, and its use-inspired research will enable the accelerated transition of modern AI approaches into benefits for society. The approach to achieve the overarching goal is to develop a bimodal interpretable multi-instance medical image classification framework by a scalable pretraining and finetuning approach. The framework consists of bimodal prototype-based interpretable contrastive pretraining to learn paired image and text prototypes from imbalanced unlabeled data, and multi-instance learning by deep area-under-the-receiver-operator-curve (AUC) maximization methods to learn from imbalanced patient-level labeled data. To make contrastive pretraining scalable and robust to imbalanced data, the investigators will develop a unified framework based on partial AUC losses, which not only unifies the existing contrastive loss but also induces new advanced global contrastive losses. The team of researchers will leverage new optimization tools and develop improved stochastic algorithms with mathematical guarantee without dependence on the large batch size of existing methods. To make multi-instance learning scalable and robust to imbalanced data, the investigators propose efficient stochastic algorithms for multi-instance deep AUC maximization by developing stochastic pooling operations from the lens of multi-level compositional optimization. The investigators will not only employ standard performance metrics for evaluation but will also leverage the domain expertise from radiologists to evaluate model performance and interpretability. The investigators will disseminate results through publications, open-source software, tutorials, workshops, and course materials, additionally engaging in outreach initiatives to enhance STEM learning and foster greater interest in the field.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目着重于创建更智能的人工智能（AI）系统，以更好地理解和分析复杂的医学图像，例如来自患者的多次扫描。传统方法取得了一些成功，但是在处理罕见疾病的情况下面临挑战，并提供了医生和患者可以轻松理解的解释。该项目旨在开发一种现代的AI方法，该方法通过利用大量医学图像和医生的注释来克服这些局限性，而与它们有关的特定健康状况如何。研究团队将应对各种挑战，以使AI系统更可扩展，可解释和健壮。这个创新的项目将向医务人员提供值得信赖的AI驱动诊断工具，从而加快复杂医疗图像的诊断过程。由于其基础研究可能会对各种应用产生影响，因此在AI研究中，该项目的影响将广泛地感受到，其使用启发的研究将使现代AI方法加速过渡到对社会的利益。 实现总体目标的方法是通过可扩展的预处理和填充方法来开发双峰解释的多企业医学图像分类框架。该框架包括基于双峰原型的可解释的对比度预处理，以从不平衡的未标记数据中学习配对的图像和文本原型，以及通过深度领域的接收者 - 操作员曲线（AUC）最大化方法的多启示性学习，以从不平衡的患者标记的标记标记的数据中学习。为了使对比预处理可扩展和强大的数据，研究人员将基于部分AUC损失开发统一的框架，这不仅统一了现有的对比损失，而且还会引起新的先进的全球对比度损失。研究人员团队将利用新的优化工具，并通过数学保证开发改进的随机算法，而不必依赖现有方法的批次大小。为了使多个数据可扩展和强大的数据对数据不平衡，研究人员提出了有效的随机算法，用于通过开发从多层组成优化的镜头开发随机合并操作，以最大程度地最大化多效性AUC最大化。研究人员不仅将采用标准绩效指标进行评估，还将利用放射科医生的域专业知识来评估模型性能和解释性。调查人员将通过出版物，开源软件，教程，讲习班和课程材料来传播结果，并还参与宣传计划，以增强STEM学习并促进对现场的更大兴趣。该奖项反映了NSF的法定任务，并通过使用该基金会的知识分子功能和广泛的影响来评估NSF的法定任务，并被视为值得的支持。