Machine learning and artificial intelligence research for clinical medical image processing

临床医学图像处理的机器学习和人工智能研究

• 批准号: 10697075
• 负责人: Sameer Antani
• 金额: $ 203.17万
• 依托单位: NATIONAL LIBRARY OF MEDICINE
• 依托单位国家: 美国
• 资助国家: 美国
• 起止时间: 至
• 项目状态: 未结题
• 来源: https://reporter.nih.gov/project-details/10697075
• 关键词: Ablation; Acetic Acids; Active Learning; Address; Agreement; Anatomy; Appearance; Area; Artificial Intelligence; Attention; Benchmarking; Biomedical Research; Biopsy; COVID-19 detection; Calibration; Cardiac; Cardiovascular Diseases; Cervical; Cervix Uteri; Characteristics; Classification; Clinical; Clinical assessments; Code; Collaborations; Computer software; Custom; Data; Data Science; Data Set; Decision Making; Detection; Diagnosis; Diagnostic; Disease; Echocardiography; Eligibility Determination; Ensure; Exhibits; Goals; Grain; HIV; Health; Histology; Histopathology; Human; Image; Image Analysis; Image Enhancement; Imagery; Imaging Device; Imaging problem; Impairment; Label; Lateral; Learning; Lesion; Location; Machine Learning; Malignant - descriptor; Malignant Neoplasms; Malignant neoplasm of cervix uteri; Malocclusion; Manuals; Maps; Measurement; Measures; Medical Imaging; Metadata; Methodology; Methods; Modality; Modeling; Morphology; Oral cavity; Performance; Play; Population; Population Heterogeneity; Property; Real-Time Systems; Reference Standards; Reporting; Reproducibility; Research; Role; Sampling; Semantics; Series; Sickle Cell Anemia; Supervision; System; Techniques; Thoracic Radiography; Time; Tissues; Training; Trust; Tuberculosis; Uncertainty; United States National Institutes of Health; Variant; Vision; Visual; Work; X-Ray Medical Imaging; artificial intelligence algorithm; automated segmentation; base; biomedical imaging; bone; cancer imaging; case control; clinical decision-making; clinical diagnostics; clinical practice; clinically significant; comorbidity; computer-assisted diagnostics; convolutional neural network; craniofacial; deep learning; deep learning algorithm; design; diagnostic tool; digital; effective therapy; experimental study; federated learning; human-in-the-loop; image processing; image registration; impression; improved; indexing; innovation; interest; learning network; learning strategy; loss of function; machine learning method; machine learning model; malignant mouth neoplasm; mortality; novel; premalignant; radiologist; soft tissue; surrogate decision making; task analysis; time use; tool; treatment planning

1. Machine Learning and Artificial Intelligence for Biomedical Images Automated computer-aided diagnostic (CADx) tools driven by ML/AI methods based on deep learning (DL) are designed to detect and differentiate disease in medical images to improve automated disease prediction and add efficiencies to human performance. Toward this, we focused our research on various medical image analysis tasks such as quality assessment, image enhancement, region of interest detection and segmentation, image classification and prediction interpretation. Several advances were made to address these topics through applications for diseases of interest. Novel works done this year include image quality assessment for cervical and oral cavity images and echocardiography images and videos. We also developed a novel unsupervised registration method for cervical cancer image sequences which resulted in a stabilized sequence toward improving visual (or automated) assessment of lesions. We developed a variety of novel ML methods and learning strategies toward improving their prediction performance. These included ensemble learning techniques which provide benefits from combining the predictions from different models and result in improved generalizability and overall accuracy. Other improvements in learning strategies were modality-specific pretraining of deep models. We also considered various other learning strategies, such as unsupervised, semi-supervised, self-supervised, deep-metric, multiple instance, and federated learning, to overcome small data set size, inadequate expert annotated labels, and case-control imbalance. Automated predictions for medical diagnostic applications demand uncertainty quantification to gain user trust which traditional ML models do not directly provide. Further, the training data might not contain the extent of disease exhibited by different populations and disease comorbidities. Therefore, we developed techniques to measure uncertainty and use human-in-the-loop expertise to actively learn new information using an open-world learning strategy to improve prediction capability. Case-control imbalance is well-known in medical image classification thereby biasing the predictions toward the majority class. We contributed toward advances in model calibration for alleviating these effects. We also benchmarked various state-of-the-art loss functions which are used in ML model training, systematically analyzed model performance, and proposed improved loss function selection strategies to counter prediction bias effects. 2. Disease-based ML/AI Research All software codes and data were made publicly available where possible. Chest X-ray bone suppression: Automated bone suppression methods would increase soft tissue visibility in chest X-rays (CXRs) and enhance automated disease detection. We developed DeBoNet, a DL algorithm to suppress bones in frontal CXRs. The DeBoNet was then applied to case and control standard digital CXR images. We observed that the model trained on bone-suppressed CXRs significantly outperformed the model trained on non-bone-suppressed images in detecting COVID-19 manifestations. Cardiovascular disease: Automated echocardiography (echo) analysis is benefited through use of machine learning for tasks such as image quality assessment, view classification, cardiac region segmentation, and quantification of diagnostic indices. We proposed a novel and efficient DL-based real-time system for echo analysis and quantification. It uses a self-supervised modality-specific representation. We evaluated the proposed system using four echo datasets. Cardiac indices extracted by the system had high agreement with experts. We also developed an open world active learning approach for echo view classification, where the network identifies images of unknown views. The system alerts the users to label unseen samples which are then integrated into the model thereby increasing the classifier robustness. Tuberculosis: Automated segmentation of tuberculosis (TB)-consistent lesions in CXRs using DL methods can help reduce radiologist effort and supplement clinical decision-making. In the first study of its kind, we evaluated the benefits of using fine-grained annotations of TB-consistent lesions toward training ensembles for semantically segmenting TB-consistent lesions in both original and bone-suppressed frontal CXRs. Results showed that the stacking ensemble demonstrated superior segmentation performance. In a separate study, we investigated the benefits of selecting an appropriate loss function and quantifying uncertainty in predictions for segmenting TB manifestations in CXRs. Highly uncertain cases are referred to an expert thereby adding reliability to the classifier. We were the first to also analyze lateral CXRs using an ensemble of modality-specific convolutional neural networks (CNN) and vision transformer models (ViT) and obtained significantly superior performance which was verified using attention maps to highlight the discriminative image regions. Cervical cancer: Colposcopic appearance is often evaluated based on static images that do not reveal the dynamics of acetowhitening. We compared the accuracy and reproducibility of colposcopic impression based on a single image at 1 minute after application of acetic acid versus a time-series of 17 sequential images over two minutes. Use of the time-series increased the proportion of images classified as normal, regardless of histology. However, substantial variation exists in visual assessment of colposcopic images using 17-image time series. For ML-based image evaluation, as a first step, we developed an image registration method to automatically spatially align dynamic images without the need for a manually-provided reference standard which improved over previously reported results. Cervical tissue ablation is an effective treatment approach for excising high-grade precancerous lesions. Following our previous work that automatically determined if a cervix was eligible for ablative treatment based on visual characteristics presented in the image, we investigated the use of an image augmenter followed by a customized classification CNN to overcome the challenges due to insufficient training data. We built the image augmenter using a CycleGAN model that was trained using three different datasets to ensure that the augmented images contain clinically significant morphological features. We gained a performance improvement in treatability eligibility classification. Oral cavity malignant lesion analysis: Oral cavity cancer is a common cancer that can result in significant impairments, and there is high mortality for the advanced stage. The final diagnosis is confirmed through histopathology, however high variability is observed among human experts in determining if a subject needs biopsy and identifying the correct biopsy location. Further, the disease can occur in different parts of the oral cavity. Toward developing an ML-based method that can help address these problems and reduce downstream classification errors, we automatically identify, with high accuracy, different anatomical sites in the oral cavity on the images that are verified using class activation maps obtained from both correct and incorrect predictions. Noting that a ruler is placed near a suspected lesion to indicate its location and as a physical size reference, we evaluated the performance of two deep-learning networks: ResNeSt and ViT, to automatically identify images with rulers. The findings were verified using heatmaps generated using three saliency methods. We also developed an automatic method for extracting the measurement information on the ruler which can help measure the lesion size. Our method is resilient to various ruler styles, visibility completeness, and overall image quality.

1。生物医学图像的机器学习和人工智能 由基于深度学习的ML/AI方法驱动的自动计算机辅助诊断（CADX）工具旨在检测和区分医学图像中的疾病，以改善自动化疾病预测并为人类绩效提高效率。为此，我们将研究集中在各种医学图像分析任务上，例如质量评估，图像增强，感兴趣的检测区域检测和细分区域，图像分类和预测解释。通过应用程序疾病的申请解决了这些主题，已取得了一些进步。今年完成的新颖作品包括宫颈和口腔图像的图像质量评估以及超声心动图和视频。我们还开发了一种用于宫颈癌图像序列的新型无监督的登记方法，该方法导致了稳定的序列，以改善病变的视觉（或自动化）评估。我们开发了各种新型的ML方法和学习策略来改善其预测性能。这些包括合奏学习技术，从结合不同模型的预测并提高了普遍性和整体准确性，从而为您提供了好处。学习策略的其他改进是深层模型的特定于模式的预读。我们还考虑了其​​他各种学习策略，例如无监督，半监督，自我监督，深度，多个实例和联合学习，以克服小数据集大小，专家注释的标签不足以及病例对照不平衡。对医学诊断应用的自动预测需要不确定性量化，以获得传统ML模型无法直接提供的用户信任。此外，训练数据可能不包含不同人群和疾病合并症所表现出的疾病程度。因此，我们开发了用于衡量不确定性的技术，并使用开放世界的学习策略来积极学习新信息，以提高预测能力。病例对照失衡在医学图像分类中众所周知，从而偏向于多数类别的预测。我们为减轻这些影响的模型校准方面的进步做出了贡献。我们还通过在ML模型训练中使用，系统分析的模型性能以及提出改进的损失函数选择策略以应对预测偏见效应的各种最新损失函数。 2。基于疾病的ML/AI研究 在可能的情况下，所有软件代码和数据均可公开使用。 胸部X射线骨抑制：自动抑制骨抑制方法将增加胸部X射线（CXR）的软组织可见性并增强自动疾病检测。我们开发了DEBONET，一种DL算法来抑制额叶CXR中的骨骼。然后将Debonet应用于案例和控制标准数字CXR图像。我们观察到，在骨抑制的CXR上训练的模型显着超过了在检测COVID-19表现时在非骨抑制图像上训练的模型。 心血管疾病：自动超声心动图（ECHO）分析通过使用机器学习来使图像质量评估，视图分类，心脏区域细分以及诊断指标的量化等任务受益。我们提出了一种新型且有效的基于DL的实时系统，用于回声分析和定量。它使用特定于特定于模式的表示。我们使用四个回声数据集评估了提出的系统。该系统提取的心脏指数与专家一致。我们还为回声视图分类开发了一种开放世界积极的学习方法，网络标识了未知视图的图像。该系统提醒用户标记看不见的样本，然后将其集成到模型中，从而增加了分类器的鲁棒性。 结核病：使用DL方法在CXR中持续的结核病（TB）自动分割可以帮助减少放射科医生的努力并补充临床决策。在对此类研究的第一项研究中，我们评估了使用TB一致性病变的细粒注释来训练集合的益处，以在原始和骨抑制的额叶CXR中对语义分割TB一致性病变进行训练集合。结果表明，堆叠合奏表现出了出色的分割性能。在另一项研究中，我们研究了选择适当的损失函数的好处，并量化了CXR中TB表现的预测中的不确定性。高度不确定的案例被转交给专家，从而为分类器增加可靠性。我们也是第一个使用模态特异性卷积神经网络（CNN）和视觉变压器模型（VIT）的集合来分析侧向CXR的人，并获得了明显优越的性能，并使用注意图进行了验证，以突出歧视性图像区域。 宫颈癌：基于无法揭示乙腹动力学的静态图像，经常评估阴道镜外观。我们比较了乙酸后1分钟基于单个图像的阴道镜印象的准确性和可重复性，而不是在两分钟内的17个顺序图像的时间序列。时间序列的使用增加了分类为正常的图像的比例，无论组织学如何。然而，使用17图段时间序列对阴道镜图像的视觉评估存在实质性变化。对于基于ML的图像评估，作为第一步，我们开发了一种图像注册方法，可以自动在空间上对齐动态图像，而无需手动提供的参考标准，这比先前报道的结果改进了。 宫颈组织消融是一种有效的治疗方法，用于切除高级癌前病变。在我们以前的工作后，自动确定了基于图像中介绍的视觉特征的子宫颈是否有资格进行消融治疗，我们研究了使用图像增强器的使用，然后使用定制的分类CNN来克服由于训练数据不足而克服挑战。我们使用Cyclean模型构建了图像增强器，该模型使用三个不同的数据集进行了训练，以确保增强图像包含临床上重要的形态特征。我们获得了可治疗性资格分类的性能提高。 口腔恶性病变分析：口腔癌是一种常见的癌症，可能导致重大损害，并且晚期死亡率很高。最终诊断是通过组织病理学确认的，但是在确定受试者是否需要活检并识别正确的活检位置的人类专家中观察到高度差异。此外，该疾病可能发生在口腔的不同部位。为了开发一种基于ML的方法，该方法可以帮助解决这些问题并减少下游分类错误，我们会以高精度自动识别图像中口腔中的不同解剖位点，这些解剖位点使用从正确和错误的预测获得的类激活图验证的图像验证。我们指出，将尺子放置在可疑病变附近，以表明其位置，作为物理大小参考，我们评估了两个深度学习网络的性能：Resnest和vit，以自动与统治者一起自动识别图像。使用使用三种显着性方法生成的热图验证了这些发现。我们还开发了一种自动方法来提取有关标尺的测量信息，该信息可以帮助测量病变的大小。我们的方法对各种尺子样式，可见性完整性和整体图像质量具有弹性。