Machine learning and artificial intelligence research for clinical medical image processing

临床医学图像处理的机器学习和人工智能研究

• 批准号: 10697075
• 负责人: Sameer Antani
• 金额: $ 203.17万
• 依托单位: NATIONAL LIBRARY OF MEDICINE
• 依托单位国家: 美国
• 资助国家: 美国
• 起止时间: 至
• 项目状态: 未结题
• 来源: https://reporter.nih.gov/project-details/10697075
• 关键词: Ablation; Acetic Acids; Active Learning; Address; Agreement; Anatomy; Appearance; Area; Artificial Intelligence; Attention; Benchmarking; Biomedical Research; Biopsy; COVID-19 detection; Calibration; Cardiac; Cardiovascular Diseases; Cervical; Cervix Uteri; Characteristics; Classification; Clinical; Clinical assessments; Code; Collaborations; Computer software; Custom; Data; Data Science; Data Set; Decision Making; Detection; Diagnosis; Diagnostic; Disease; Echocardiography; Eligibility Determination; Ensure; Exhibits; Goals; Grain; HIV; Health; Histology; Histopathology; Human; Image; Image Analysis; Image Enhancement; Imagery; Imaging Device; Imaging problem; Impairment; Label; Lateral; Learning; Lesion; Location; Machine Learning; Malignant - descriptor; Malignant Neoplasms; Malignant neoplasm of cervix uteri; Malocclusion; Manuals; Maps; Measurement; Measures; Medical Imaging; Metadata; Methodology; Methods; Modality; Modeling; Morphology; Oral cavity; Performance; Play; Population; Population Heterogeneity; Property; Real-Time Systems; Reference Standards; Reporting; Reproducibility; Research; Role; Sampling; Semantics; Series; Sickle Cell Anemia; Supervision; System; Techniques; Thoracic Radiography; Time; Tissues; Training; Trust; Tuberculosis; Uncertainty; United States National Institutes of Health; Variant; Vision; Visual; Work; X-Ray Medical Imaging; artificial intelligence algorithm; automated segmentation; base; biomedical imaging; bone; cancer imaging; case control; clinical decision-making; clinical diagnostics; clinical practice; clinically significant; comorbidity; computer-assisted diagnostics; convolutional neural network; craniofacial; deep learning; deep learning algorithm; design; diagnostic tool; digital; effective therapy; experimental study; federated learning; human-in-the-loop; image processing; image registration; impression; improved; indexing; innovation; interest; learning network; learning strategy; loss of function; machine learning method; machine learning model; malignant mouth neoplasm; mortality; novel; premalignant; radiologist; soft tissue; surrogate decision making; task analysis; time use; tool; treatment planning

1. Machine Learning and Artificial Intelligence for Biomedical Images Automated computer-aided diagnostic (CADx) tools driven by ML/AI methods based on deep learning (DL) are designed to detect and differentiate disease in medical images to improve automated disease prediction and add efficiencies to human performance. Toward this, we focused our research on various medical image analysis tasks such as quality assessment, image enhancement, region of interest detection and segmentation, image classification and prediction interpretation. Several advances were made to address these topics through applications for diseases of interest. Novel works done this year include image quality assessment for cervical and oral cavity images and echocardiography images and videos. We also developed a novel unsupervised registration method for cervical cancer image sequences which resulted in a stabilized sequence toward improving visual (or automated) assessment of lesions. We developed a variety of novel ML methods and learning strategies toward improving their prediction performance. These included ensemble learning techniques which provide benefits from combining the predictions from different models and result in improved generalizability and overall accuracy. Other improvements in learning strategies were modality-specific pretraining of deep models. We also considered various other learning strategies, such as unsupervised, semi-supervised, self-supervised, deep-metric, multiple instance, and federated learning, to overcome small data set size, inadequate expert annotated labels, and case-control imbalance. Automated predictions for medical diagnostic applications demand uncertainty quantification to gain user trust which traditional ML models do not directly provide. Further, the training data might not contain the extent of disease exhibited by different populations and disease comorbidities. Therefore, we developed techniques to measure uncertainty and use human-in-the-loop expertise to actively learn new information using an open-world learning strategy to improve prediction capability. Case-control imbalance is well-known in medical image classification thereby biasing the predictions toward the majority class. We contributed toward advances in model calibration for alleviating these effects. We also benchmarked various state-of-the-art loss functions which are used in ML model training, systematically analyzed model performance, and proposed improved loss function selection strategies to counter prediction bias effects. 2. Disease-based ML/AI Research All software codes and data were made publicly available where possible. Chest X-ray bone suppression: Automated bone suppression methods would increase soft tissue visibility in chest X-rays (CXRs) and enhance automated disease detection. We developed DeBoNet, a DL algorithm to suppress bones in frontal CXRs. The DeBoNet was then applied to case and control standard digital CXR images. We observed that the model trained on bone-suppressed CXRs significantly outperformed the model trained on non-bone-suppressed images in detecting COVID-19 manifestations. Cardiovascular disease: Automated echocardiography (echo) analysis is benefited through use of machine learning for tasks such as image quality assessment, view classification, cardiac region segmentation, and quantification of diagnostic indices. We proposed a novel and efficient DL-based real-time system for echo analysis and quantification. It uses a self-supervised modality-specific representation. We evaluated the proposed system using four echo datasets. Cardiac indices extracted by the system had high agreement with experts. We also developed an open world active learning approach for echo view classification, where the network identifies images of unknown views. The system alerts the users to label unseen samples which are then integrated into the model thereby increasing the classifier robustness. Tuberculosis: Automated segmentation of tuberculosis (TB)-consistent lesions in CXRs using DL methods can help reduce radiologist effort and supplement clinical decision-making. In the first study of its kind, we evaluated the benefits of using fine-grained annotations of TB-consistent lesions toward training ensembles for semantically segmenting TB-consistent lesions in both original and bone-suppressed frontal CXRs. Results showed that the stacking ensemble demonstrated superior segmentation performance. In a separate study, we investigated the benefits of selecting an appropriate loss function and quantifying uncertainty in predictions for segmenting TB manifestations in CXRs. Highly uncertain cases are referred to an expert thereby adding reliability to the classifier. We were the first to also analyze lateral CXRs using an ensemble of modality-specific convolutional neural networks (CNN) and vision transformer models (ViT) and obtained significantly superior performance which was verified using attention maps to highlight the discriminative image regions. Cervical cancer: Colposcopic appearance is often evaluated based on static images that do not reveal the dynamics of acetowhitening. We compared the accuracy and reproducibility of colposcopic impression based on a single image at 1 minute after application of acetic acid versus a time-series of 17 sequential images over two minutes. Use of the time-series increased the proportion of images classified as normal, regardless of histology. However, substantial variation exists in visual assessment of colposcopic images using 17-image time series. For ML-based image evaluation, as a first step, we developed an image registration method to automatically spatially align dynamic images without the need for a manually-provided reference standard which improved over previously reported results. Cervical tissue ablation is an effective treatment approach for excising high-grade precancerous lesions. Following our previous work that automatically determined if a cervix was eligible for ablative treatment based on visual characteristics presented in the image, we investigated the use of an image augmenter followed by a customized classification CNN to overcome the challenges due to insufficient training data. We built the image augmenter using a CycleGAN model that was trained using three different datasets to ensure that the augmented images contain clinically significant morphological features. We gained a performance improvement in treatability eligibility classification. Oral cavity malignant lesion analysis: Oral cavity cancer is a common cancer that can result in significant impairments, and there is high mortality for the advanced stage. The final diagnosis is confirmed through histopathology, however high variability is observed among human experts in determining if a subject needs biopsy and identifying the correct biopsy location. Further, the disease can occur in different parts of the oral cavity. Toward developing an ML-based method that can help address these problems and reduce downstream classification errors, we automatically identify, with high accuracy, different anatomical sites in the oral cavity on the images that are verified using class activation maps obtained from both correct and incorrect predictions. Noting that a ruler is placed near a suspected lesion to indicate its location and as a physical size reference, we evaluated the performance of two deep-learning networks: ResNeSt and ViT, to automatically identify images with rulers. The findings were verified using heatmaps generated using three saliency methods. We also developed an automatic method for extracting the measurement information on the ruler which can help measure the lesion size. Our method is resilient to various ruler styles, visibility completeness, and overall image quality.

1.生物医学图像的机器学习和人工智能 由基于深度学习 (DL) 的 ML/AI 方法驱动的自动计算机辅助诊断 (CADx) 工具旨在检测和区分医学图像中的疾病，以改进自动疾病预测并提高人类表现的效率。为此，我们将研究重点放在各种医学图像分析任务上，例如质量评估、图像增强、感兴趣区域检测和分割、图像分类和预测解释。通过针对感兴趣的疾病的应用，在解决这些主题方面取得了一些进展。今年完成的新工作包括颈部和口腔图像的图像质量评估以及超声心动图图像和视频。我们还开发了一种新型的宫颈癌图像序列无监督配准方法，该方法产生稳定的序列，从而改善病变的视觉（或自动）评估。我们开发了各种新颖的机器学习方法和学习策略，以提高其预测性能。其中包括集成学习技术，该技术通过结合不同模型的预测来提供好处，并提高通用性和整体准确性。学习策略的其他改进是深度模型的特定模态预训练。我们还考虑了各种其他学习策略，例如无监督、半监督、自监督、深度度量、多实例和联合学习，以克服数据集大小小、专家注释标签不足和病例控制不平衡的问题。医疗诊断应用的自动预测需要不确定性量化，以获得传统机器学习模型无法直接提供的用户信任。此外，训练数据可能不包含不同人群表现出的疾病程度和疾病合并症。因此，我们开发了测量不确定性的技术，并利用人机交互的专业知识，使用开放世界的学习策略主动学习新信息，以提高预测能力。病例对照不平衡在医学图像分类中是众所周知的，从而使预测偏向于多数类别。我们为模型校准的进步做出了贡献，以减轻这些影响。我们还对机器学习模型训练中使用的各种最先进的损失函数进行了基准测试，系统地分析了模型性能，并提出了改进的损失函数选择策略来对抗预测偏差效应。 2. 基于疾病的机器学习/人工智能研究 所有软件代码和数据均尽可能公开。 胸部 X 射线骨抑制：自动骨抑制方法将增加胸部 X 光 (CXR) 中软组织的可见度，并增强自动化疾病检测。我们开发了 DeBoNet，这是一种用于抑制额叶 CXR 中骨骼的深度学习算法。然后将 DeBoNet 应用于病例和对照标准数字 CXR 图像。我们观察到，在检测 COVID-19 表现方面，在骨抑制 CXR 上训练的模型明显优于在非骨抑制图像上训练的模型。 心血管疾病：通过使用机器学习执行图像质量评估、视图分类、心脏区域分割和诊断指标量化等任务，自动超声心动图 (echo) 分析受益匪浅。我们提出了一种新颖且高效的基于深度学习的实时系统，用于回声分析和量化。它使用自我监督的特定模态表示。我们使用四个回波数据集评估了所提出的系统。系统提取的心脏指标与专家的一致性较高。我们还开发了一种用于回波视图分类的开放世界主动学习方法，其中网络识别未知视图的图像。系统提醒用户标记未见过的样本，然后将其集成到模型中，从而提高分类器的鲁棒性。 结核病：使用深度学习方法对 CXR 中结核病 (TB) 一致的病变进行自动分割，有助于减少放射科医生的工作量并补充临床决策。在此类研究中，我们评估了使用 TB 一致病变的细粒度注释来训练集成，以便在原始和骨抑制额叶 CXR 中对 TB 一致病变进行语义分割。结果表明，堆叠集成表现出优异的分割性能。在另一项研究中，我们研究了选择适当的损失函数和量化 CXR 中结核病表现分段预测的不确定性的好处。高度不确定的情况会被转介给专家，从而增加分类器的可靠性。我们也是第一个使用特定模态的卷积神经网络 (CNN) 和视觉变换模型 (ViT) 的集合来分析横向 CXR，并获得了显着优越的性能，并使用注意图突出显示有区别的图像区域来验证这一性能。 宫颈癌：阴道镜外观通常根据静态图像进行评估，这些图像不能揭示醋酸美白的动态。我们比较了基于施加醋酸后 1 分钟的单个图像的阴道镜印模的准确性和可重复性与两分钟内 17 个连续图像的时间序列。无论组织学如何，时间序列的使用增加了分类为正常的图像的比例。然而，使用 17 个图像时间序列对阴道镜图像进行视觉评估时存在很大差异。对于基于机器学习的图像评估，作为第一步，我们开发了一种图像配准方法，可以自动在空间上对齐动态图像，而不需要手动提供的参考标准，这比之前报告的结果有所改进。 宫颈组织消融是切除高级别癌前病变的有效治疗方法。继我们之前根据图像中呈现的视觉特征自动确定子宫颈是否适合进行消融治疗的工作之后，我们研究了使用图像增强器和定制的分类 CNN 来克服由于训练数据不足而带来的挑战。我们使用 CycleGAN 模型构建了图像增强器，该模型使用三个不同的数据集进行训练，以确保增强图像包含临床上重要的形态特征。我们在可治疗性资格分类方面取得了性能改进。 口腔恶性病变分析：口腔癌是一种常见的癌症，可造成明显的损害，晚期死亡率较高。最终诊断是通过组织病理学确认的，然而，在确定受试者是否需要活检和确定正确的活检位置时，人类专家之间观察到很大的差异。此外，该疾病可以发生在口腔的不同部位。为了开发一种基于机器学习的方法来帮助解决这些问题并减少下游分类错误，我们在图像上以高精度自动识别口腔中的不同解剖部位，并使用从正确和错误获得的类激活图进行验证预测。注意到将尺子放置在可疑病变附近以指示其位置并作为物理尺寸参考，我们评估了两个深度学习网络：ResNeSt 和 ViT 的性能，以自动识别带有尺子的图像。使用三种显着性方法生成的热图验证了研究结果。我们还开发了一种自动提取尺子上测量信息的方法，可以帮助测量病变大小。我们的方法可以适应各种标尺样式、可见性完整性和整体图像质量。