Nucleus subtype classification using inter-modality learning

Understanding the way cells communicate, co-locate, and interrelate is essential to understanding human physiology. Hematoxylin and eosin (H&E) staining is ubiquitously available both for clinical studies and research. The Colon Nucleus Identification and Classification (CoNIC) Challenge has recently innovated on robust artificial intelligence labeling of six cell types on H&E stains of the colon. However, this is a very small fraction of the number of potential cell classification types. Specifically, the CoNIC Challenge is unable to classify epithelial subtypes (progenitor, endocrine, goblet), lymphocyte subtypes (B, helper T, cytotoxic T), or connective subtypes (fibroblasts, stromal). In this paper, we propose to use inter-modality learning to label previously un-labelable cell types on virtual H&E. We leveraged multiplexed immunofluorescence (MxIF) histology imaging to identify 14 subclasses of cell types. We performed style transfer to synthesize virtual H&E from MxIF and transferred the higher density labels from MxIF to these virtual H&E images. We then evaluated the efficacy of learning in this approach. We identified helper T and progenitor nuclei with positive predictive values of 0.34 ± 0.15 (prevalence 0.03 ± 0.01) and 0.47 ± 0.1 (prevalence 0.07 ± 0.02) respectively on virtual H&E. This approach represents a promising step towards automating annotation in digital pathology.

理解细胞交流、共定位和相互关联的方式对于理解人体生理学至关重要。苏木精 - 伊红（H&E）染色在临床研究和科研中普遍使用。结肠细胞核识别与分类（CoNIC）挑战赛近期在对结肠H&E染色中的六种细胞类型进行稳健的人工智能标记方面有所创新。然而，这只是潜在细胞分类类型数量中很小的一部分。具体而言，CoNIC挑战赛无法对上皮亚型（祖细胞、内分泌细胞、杯状细胞）、淋巴细胞亚型（B细胞、辅助性T细胞、细胞毒性T细胞）或结缔组织亚型（成纤维细胞、基质细胞）进行分类。在本文中，我们提议使用模态间学习来标记虚拟H&E上以前无法标记的细胞类型。我们利用多重免疫荧光（MxIF）组织学成像来识别14种细胞类型的亚类。我们进行了风格迁移，从MxIF合成虚拟H&E，并将更高密度的标记从MxIF转移到这些虚拟H&E图像上。然后我们评估了这种方法的学习效果。我们在虚拟H&E上识别出辅助性T细胞核和祖细胞核，其阳性预测值分别为0.34±0.15（患病率0.03±0.01）和0.47±0.1（患病率0.07±0.02）。这种方法代表了在数字病理学中实现自动注释的有希望的一步。