Guiding humans to create better labeled datasets for machine learning in biomedical research

指导人类为生物医学研究中的机器学习创建更好的标记数据集

基本信息

批准号：
10298684
负责人：
Lee Cooper
金额：
$ 43.09万
依托单位：
NORTHWESTERN UNIVERSITY AT CHICAGO
依托单位国家：
美国
项目类别：
财政年份：
2021
资助国家：
美国
起止时间：
2021-09-01 至 2025-05-31
项目状态：
未结题

来源：
https://reporter.nih.gov/project-details/10298684
关键词：
Active Learning Address Algorithms Bayesian neural network Biological Biomedical Research Clinical Informatics Clinical Research Clinical Trials Code Collaborations Communities Computer Systems Computer software Data Data Scientist Data Set Databases Environment Face Fetal health Funding Growth High Performance Computing Histologic Human Image Institutes Institution K-Series Research Career Programs Knowledge Label Lead Learning Machine Learning Maternal Health Measurement Methodology Methods Modeling Natural Language Processing Pathologist Pathology Pattern Performance Perinatal Placenta Process Recording of previous events Reproducibility Research Research Personnel Resources Sampling Science Site Software Framework Software Tools Source Structure Tissue imaging Training Work algorithm training base cloud platform cohort computing resources deep learning deep learning algorithm digital pathology experience feature extraction hands-on learning human-in-the-loop improved large datasets learning strategy machine learning algorithm malignant breast neoplasm multidimensional data novel strategies open source pathology imaging public health relevance repository simulation software development tool tool development unsupervised learning whole slide imaging

项目摘要

PROJECT SUMMARY / ABSTRACT Machine learning (ML) has seen tremendous advances in the past decade, fueled by growth in computing and the availability of large labeled datasets. While the impact of these advances on clinical and biomedical research are potentially significant, these applications face unique challenges due to the difficulty in acquiring labels from biomedical experts. Furthermore, ML algorithms often fail to generalize across institutions or datasets due to measurement biases (e.g. MR scanners) or intrinsic demographic or biological differences between cohorts / datasets which limits their impact in biomedical science. This proposal will develop new methodology and open-source software that biomedical data scientists can use with their applications to 1. Improve data labeling by identifying the best samples for labeling that provide the most benefit for training ML algorithms; 2. Improve generalization of ML models across institutes; and 3. Perform this work on scalable cloud platforms. We will first explore how to improve upon methods known as active learning that interactively construct labeled datasets by having an algorithm select samples that address its weaknesses and present these samples to an expert for labeling. We will then investigate how these samples can be selected to improve the performance of ML algorithms across multiple institutions by learning robust patterns that are not specific to any one site. Finally, we will develop an extendable software framework that developers can integrate into their own applications to take advantage of these methods, and that can operate on cloud platforms to support scalable analysis of large datasets. This work will be developed through a combination of simulation studies using a unique repository of over 280,000 human markups of digital pathology images at multiple institutions, and also user studies of the developed software frameworks focused on applications in perinatal pathology and the human placenta. The software tools will impact a broad variety of biomedical applications beyond pathology where data labeling and multi-institutional studies remain challenging.

项目概要/摘要在计算和技术发展的推动下，机器学习 (ML) 在过去十年中取得了巨大进步大型标记数据集的可用性。虽然这些进步对临床和生物医学的影响研究具有潜在的重要意义，但由于获取困难，这些应用面临着独特的挑战生物医学专家的标签。此外，机器学习算法通常无法跨机构或由于测量偏差（例如 MR 扫描仪）或固有的人口统计或生物学差异而产生的数据集队列/数据集之间的差异限制了它们在生物医学科学中的影响。该提案将开发新的生物医学数据科学家可以在其应用程序中使用的方法和开源软件 1. 通过识别为训练 ML 提供最大益处的最佳标记样本来改进数据标记算法； 2. 提高跨机构机器学习模型的泛化能力； 3. 在可扩展的基础上执行这项工作云平台。我们将首先探讨如何改进称为主动学习的方法，该方法以交互方式进行通过让算法选择解决其弱点并呈现的样本来构建标记数据集将这些样品交给专家进行标记。然后我们将研究如何选择这些样本通过学习非结构化的稳健模式，提高跨多个机构的 ML 算法的性能特定于任何一个站点。最后，我们将开发一个可扩展的软件框架，开发人员可以集成到自己的应用程序中以利用这些方法，并且可以在云上运行支持大型数据集可扩展分析的平台。这项工作将通过以下方面的结合来开发使用包含超过 280,000 个数字病理图像人工标记的独特存储库进行模拟研究多个机构，以及开发的软件框架的用户研究，重点关注应用程序围产期病理学和人类胎盘。这些软件工具将影响广泛的生物医学领域病理学之外的应用，其中数据标记和多机构研究仍然具有挑战性。