SCH: Blazing Data Trails: Digital Pathology and Specialist Attention

SCH：惊人的数据线索：数字病理学和专家关注

• 批准号: 2123920
• 负责人: Dimitrios Samaras
• 金额: $ 120万
• 依托单位: SUNY at Stony Brook
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-09-01 至 2025-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2123920&HistoricalAwards=false
• 关键词: SCH; Blazing; Data; Trails; Digital

When looking for cancer in clinical slides, pathologists move the focus of their attention around the slides in complex ways. These skilled shifts of attention are critical to how pathologists make expert diagnoses. This research program seeks to understand these shifts in attention in order to build an artificial intelligence (AI) system that that will be able to look at a slide the way a human pathologist would. Building an “AI expert pathologist,” however, requires a lot of data for it to learn, just like a pathologist needs years of training to become an expert. In order to provide the model with many examples of expert attention behavior, essential for it to make good predictions, the investigators will collect a large dataset of attention behavior from human pathologists. The human pathologists’ behavior will also serve as feedback to the AI model, enabling the AI system to model and reproduce how the human pathologists expertly sample the slides by moving their focus of attention. The investigators will also build AI-fueled tools that can predict where an expert would have focused their attention in a slide, thereby giving human pathologists feedback from the AI pathologist. The aim is to improve human accuracy of cancer diagnoses, which is paramount to improving the healthcare infrastructure of the country. The work also has the potential to improve histopathology training in medical personnel and to lead to next-generation AI models for cancer classification. The AI scientists trained through this project will be experts in building AI-tools that understand human expert performance and synergistically enhance it. A large database will be created of pathologist’s cursor-based movements during cancer interpretations, referred to as attention trajectories. These will be collected online from pathologists searching for metastatic cancer in Whole Slide Images (WSIs) of lymph nodes that were excised as part of cancer surgeries. For each WSI, one of four “diagnoses” will also be collected: negative, small, medium, or large metastases. Using a family of AI methods called imitation learning, the investigators will generate personalized as well as group prediction models of pathologist attention trajectories, applying Active Imitation Learning to real human behavior. Techniques for batch processing and pathologist-in-the-loop learning of attention trajectories will also be developed. An improvement in the efficiency and accuracy of pathology classification algorithms is expected through use of a multi-resolution approach that only processes small parts of WSIs by combining computational and human attention priors. Lastly, attention-based diagnostic aids that suggest areas to examine at higher magnification will be developed for human pathologists to use during slide interpretationThis award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

在临床幻灯片中寻找癌症时，病理学家以复杂的方式将注意力的重点移动到幻灯片周围。这些熟练的关注转变对于病理学家如何进行专家诊断至关重要。该研究计划旨在了解这些关注的转变，以建立人工智能（AI）系统，该系统将能够以人类病理学家的方式看滑动。但是，建立“ AI专家病理学家”需要大量数据才能学习，就像病理学家需要多年的培训成为专家一样。为了为模型提供许多经验注意行为的例子，对其进行良好的预测至关重要，研究人员将从人类病理学家那里收集大量的注意行为数据集。人类病理学家的行为还将作为对AI模型的反馈，使AI系统能够建模和复制人类病理学家如何通过移动注意力的重点来熟练地品尝幻灯片。调查人员还将建立AI燃料的工具，可以预测专家会在幻灯片上集中注意力的位置，从而使人类病理学家的反馈来自AI病理学家。目的是提高癌症诊断的人类准确性，这对于改善该国的医疗保健基础设施至关重要。这项工作还具有改善医疗人员的组织病理学培训，并导致下一代AI癌症模型。分类。经过该项目培训的AI科学家将是建立AI-Tools的专家，以了解人类专家的表现并协同增强它。在癌症解释期间，将创建一个大型数据库，该数据库被称为注意轨迹。这些将从病理学家在线收集，以在淋巴结的整个幻灯片图像（WSI）中寻找转移性癌症，这些淋巴结是癌症手术的一部分。对于每个WSI，还将收集四个“诊断”之一：负，小，中或大转移。使用称为模仿学习的AI方法的家族，研究人员将生成个性化的病理学家注意轨迹的个性化和小组预测模型，将主动模仿学习应用于真实的人类行为。还将开发用于批处理处理和病理学家注意轨迹的技术。通过使用多分辨率方法，仅通过结合计算和人类注意的先验来处理WSIS的一小部分，可以提高病理分类算法的效率和准确性。最后，将开发基于注意力的诊断辅助工具，这些诊断辅助工具将在幻灯片解释期间开发较高放大的领域，以供人类病理学家使用，这反映了NSF的法定任务，并被认为是值得通过基金会的智力优点和更广泛影响的评估评估标准的评估值得支持的。

项目成果

Token Sparsification for Faster Medical Image Segmentation

• DOI: 10.48550/arxiv.2303.06522
• 发表时间: 2023-03
• 期刊: Information processing in medical imaging : proceedings of the ... conference
• 作者: Lei Zhou;Huidong Liu;Joseph Bae;Junjun He;D. Samaras;P. Prasanna

Using Generated Object Reconstructions to Study Object-based Attention

使用生成的对象重建来研究基于对象的注意力

• DOI: 10.32470/ccn.2023.1685-0
• 发表时间: 2023
• 作者: Ahn S

Unsupervised Stain Decomposition via Inversion Regulation for Multiplex Immunohistochemistry Images

通过多重免疫组织化学图像的反转调节进行无监督染色分解

• 发表时间: 2023
• 期刊: Proceedings of Machine Learning Research
• 作者: Shahira Abousamra, Danielle Fassler

Characterizing Target-absent Human Attention

• DOI: 10.1109/cvprw56347.2022.00551
• 发表时间: 2022-06
• 期刊: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)
• 作者: Yupei Chen;Zhibo Yang;Souradeep Chakraborty;Sounak Mondal;Seoyoung Ahn;D. Samaras;Minh Hoai;G. Zelinsky

Precise Location Matching Improves Dense Contrastive Learning in Digital Pathology

• DOI: 10.48550/arxiv.2212.12105
• 发表时间: 2022-12
• 期刊: ArXiv
• 作者: Jingwei Zhang;S. Kapse;Ke Ma;P. Prasanna;M. Vakalopoulou;J. Saltz;D. Samaras