NSF FDA/SiR: Development of eeDAP microscopy platform software, validation data, and statistical methods to assess performance of candidate Software as a Medical Device (SaMD)

NSF FDA/SiR：开发 eeDAP 显微镜平台软件、验证数据和统计方法，以评估候选软件作为医疗设备 (SaMD) 的性能

• 批准号: 2326317
• 负责人: Kim Blenman
• 金额: $ 20万
• 依托单位: Yale University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-10-01 至 2025-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2326317&HistoricalAwards=false
• 关键词: NSF; FDA; SiR; Development; eeDAP

Computational pathology has recently exploded with the advent of sophisticated quantitative imaging microscopes and machine learning (ML) analysis software. FDA scientists have indicated that there is a need for statistical methods and relevant data for designing studies of readers skilled in the art of histology slide review (e.g., pathologists, immunologists, pathology extenders) and of AI/ML models. Therefore, this NSF/FDA Scholar-in-Residence project focuses on developing blueprints for components of reader studies in which readers interpret medical images with and without AI/ML model outputs. The work includes design of software for an FDA designed optical and digital microscope, validation data, and statistical methods used for assessment of AI/ML models.The first objective of this NSF/FDA Scholar-in-Residence project is to validate the existing MATLAB based Evaluation Environment for Digital and Analog Pathology (eeDAP) platform as a tool for reader studies and develop free open-source software tools to control the eeDAP platform. The team will build a “bridge” between the precisionFDA portal, multiple expert readers’ computers, and the local eeDAP unit. Histology slide annotation software and Python microscope hardware control scripts will be integrated using a client-server model with the use of containerization, workload orchestration, and overlay networks to create an application that is scalable across multiple devices. This will facilitate interactions with the precisionFDA portal and with experts during discussions, crowdsourcing, and teaching. The second objective of the project is to measure, account for, and develop strategies to reduce reader variability. Methods that threshold the data will be used. The methods separately evaluate agreement above and below the threshold for each member of an expert panel. The test statistic is the agreement between the end user and each expert minus the agreement between each pair of experts, averaged over the experts. Models of this test statistic will be used to generate hypothesis tests. Multiple thresholds and corresponding hypothesis tests will be treated sequentially to investigate performance across the data range. These tools are needed to facilitate reproducible, generalizable, statistically efficient, and practical device assessments in this space for use in evaluation of software planned for use in clinical care. Other diagnostic imaging tests suffer for the same challenges. Therefore, methods developed in this proposal have applicability well beyond digital pathology.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

随着复杂的定量成像显微镜和机器学习 (ML) 分析软件的出现，计算病理学最近出现了爆炸式增长，FDA 科学家表示，需要统计方法和相关数据来为精通组织学幻灯片审查领域的读者设计研究。 （例如病理学家、免疫学家、病理学扩展者）和 AI/ML 模型的研究因此，这个 NSF/FDA 常驻学者项目的重点是为读者研究的组成部分开发蓝图，其中读者可以使用或不使用 AI/ML 来解释医学图像。这项工作包括为 FDA 设计的光学和数字显微镜设计软件、验证数据以及用于评估 AI/ML 模型的统计方法。该 NSF/FDA 驻场学者项目的首要目标是验证。现有基于 MATLAB 的数字和模拟病理学评估环境 (eeDAP) 平台作为读者研究的工具，并开发免费的开源软件工具来控制 eeDAP 平台，该团队将在 precisionFDA 门户、多位专家之间搭建一座“桥梁”。读者的电脑，本地 eeDAP 单元将使用客户端-服务器模型与容器化、工作负载编排和覆盖网络进行集成，以创建可跨多个设备扩展的应用程序。该项目的第二个目标是在讨论、众包和教学过程中促进与 precisionFDA 门户和专家的互动。评估专家小组每个成员高于和低于阈值的一致性。测试统计量是最终用户和每个专家之间的一致性减去每对专家之间的一致性，将使用此测试统计量的模型。将按顺序处理多个阈值和相应的假设检验，以调查整个数据范围内的性能，以促进在该领域有效地进行可重复的、可推广的、理论上的和实用的设备评估，以用于评估计划的软件。用于临床护理。因此，该提案中开发的方法的适用性远远超出了数字病理学。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。