RI: Medium: Information Super-Resolution for Very Large Images

RI：中：超大图像的信息超分辨率

• 批准号: 2212046
• 负责人: Dimitrios Samaras
• 金额: $ 112.9万
• 依托单位: SUNY at Stony Brook
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-09-01 至 2026-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2212046&HistoricalAwards=false
• 关键词: RI; Medium; Information; Super; Resolution

Artificial intelligence and Machine Learning have in recent years been applied to the analysis of very large scale (VLS) images such as those encountered in the analysis of aerial or satellite imagery and digital histopathology, so that domain scientists can explore the data and form novel hypotheses. The use of the current state-of-the-art deep learning techniques requires vast amounts of detailed annotations (a.k.a. labels) as training data, which can be proportional to the size of the input images. Thus, it is either impossible or very expensive to acquire enough high-resolution training data. In this project, the research team will develop a methodology that uses weaker (or auxiliary) signals collected in much smaller, low-resolution images to efficiently constrain the spatial (or temporal) statistical distribution of the labels in the high-resolution image. The framework significantly reduces the human effort needed for the mundane task of annotating VLS images, which is crucial for several exciting applications to predict environmental trends and cancer treatment outcomes. The developed techniques are general, and their application will be demonstrated in two different domains involving very large images, satellite imagery and digital histopathology. In environmental applications, the ability to directly connect satellite imagery to policy-relevant metrics of interest (e.g., population trends, urbanization, biodiversity loss, etc.) would radically improve our capacity to monitor the globe. Similarly, being able to reliably extract high resolution information from whole slide images of histopathology will be highly useful for cancer research focused on the development of novel diagnostic tests and numerous precision medicine applications (e.g., patient stratification, treatment selection, prediction of disease progression, recurrence, treatment response, and disease-free survival through downstream correlations with clinical, radiologic, laboratory, molecular, pharmacologic, and outcomes data). The technical aims of the project are: i) The research team addresses the problem of super-resolving dense annotations by matching label statistics across resolutions. The general methodology for differentiable loss functions maps auxiliary constraints to high-resolution labels. Each Label Super-Resolution loss is a differentiable distance metric between a distribution and a set of statistical values; ii) The research team generalizes the concept of super-resolution to topological information (through persistent homology) and use multi-task learning to produce latent representations that can be the basis of various inference tasks; iii) In the developed framework, the research team models missing auxiliary data, heterogeneous auxiliary data, and dynamic image sets of the same area and our losses can be easily integrated in RNN/transformer architectures and adversarial learning paradigms; iv) The research team evaluates two modalities of incremental human engagement: 1) Showing the annotator the effects of their annotation choices to help develop intuition for high return areas and 2) A reinforcement learning based active learning framework that imitates how domain experts select what kinds of data to label; and v) The research team develops and evaluates ideas through a number of well-grounded applications of Label Super-Resolution.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

近年来，人工智能和机器学习已应用于非常大规模（VLS）图像的分析，例如在空中或卫星图像和数字组织病理学分析中遇到的图像，以便域科学家可以探索数据并形成新的假设。当前最新的深度学习技术的使用需要大量详细注释（又称标签）作为培训数据，这可以与输入图像的大小成比例。因此，获得足够的高分辨率培训数据是不可能的，要么非常昂贵。在该项目中，研究团队将开发一种方法，该方法使用在较小，低分辨率图像中收集的较弱（或辅助）信号，以有效地约束标签的空间（或时间）统计分布在高分辨率图像中。该框架大大减少了注释VLS图像的平凡任务所需的人类努力，这对于预测环境趋势和癌症治疗结果的几种令人兴奋的应用至关重要。开发的技术是一般的，它们的应用将在涉及非常大图像，卫星图像和数字组织病理学的两个不同领域中证明。 在环境应用中，将卫星图像直接连接到与政策相关的感兴趣指标（例如人口趋势，城市化，生物多样性损失等）的能力从根本上提高了我们监视地球的能力。同样，能够可靠地从组织病理学的整个幻灯片图像中可靠提取高分辨率信息将对癌症研究非常有用，该研究专注于开发新的诊断测试和大量精确医学应用（例如，患者分层，治疗，治疗选择，预测疾病进展，复发性，治疗，治疗，治疗，治疗，治疗，治疗，无疾病的无疾病，通过临床学，分数，分数，分数，分数，分数，分数。该项目的技术目的是：i）研究小组通过跨决议的标签统计数据来解决超出分辨的密集注释的问题。可区分损失函数的一般方法将辅助约束映射到高分辨率标签。每个标签超分辨率损失都是分布与一组统计值之间的可区分距离度量。 ii）研究团队将超分辨率的概念推广到拓扑信息（通过持续的同源性），并使用多任务学习来产生可能是各种推理任务的基础的潜在表示； iii）在开发的框架中，研究团队模拟了缺少辅助数据，异质辅助数据以及同一区域的动态图像集，并且我们的损失可以轻松地集成到RNN/Transformer架构和对抗性学习范式中； iv）研究小组评估了人类参与的两种方式：1）向注释者展示其注释选择的效果，以帮助开发高回报区域的直觉和2）基于增强学习的主动学习框架，以模仿域专家如何选择标记哪些数据类型的数据； v）研究团队通过标签超级分辨率的许多基础应用来制定和评估思想。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的知识分子优点和更广泛的影响评估的评估来支持的。

项目成果

Token Sparsification for Faster Medical Image Segmentation

• DOI: 10.48550/arxiv.2303.06522
• 发表时间: 2023-03
• 期刊: Information processing in medical imaging : proceedings of the ... conference
• 作者: Lei Zhou;Huidong Liu;Joseph Bae;Junjun He;D. Samaras;P. Prasanna

Precise Location Matching Improves Dense Contrastive Learning in Digital Pathology

• DOI: 10.48550/arxiv.2212.12105
• 发表时间: 2022-12
• 期刊: ArXiv
• 作者: Jingwei Zhang;S. Kapse;Ke Ma;P. Prasanna;M. Vakalopoulou;J. Saltz;D. Samaras

Unsupervised Stain Decomposition via Inversion Regulation for Multiplex Immunohistochemistry Images

通过多重免疫组织化学图像的反转调节进行无监督染色分解

• 发表时间: 2023
• 期刊: Proceedings of Machine Learning Research
• 作者: Shahira Abousamra, Danielle Fassler

Topology-Guided Multi-Class Cell Context Generation for Digital Pathology

用于数字病理学的拓扑引导多类细胞上下文生成

• DOI: 10.1109/cvpr52729.2023.00324
• 发表时间: 2023
• 期刊: 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR
• 作者: Abousamra, Shahira;Gupta, Rajarsi;Kurc, Tahsin;Samaras, Dimitris;Saltz, Joel;Chen, Chao
• 通讯作者: Chen, Chao

Generating Features with Increased Crop-Related Diversity for Few-Shot Object Detection

• DOI: 10.1109/cvpr52729.2023.01888
• 发表时间: 2023-04
• 期刊: 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
• 作者: Jingyi Xu;Hieu M. Le;D. Samaras