Comparing Performance Under FROC Paradigm - A Non Parametirc Approach

比较 FROC 范式下的性能 - 非参数方法

• 批准号: 7129705
• 负责人: DAVID GUR
• 金额: $ 32.89万
• 依托单位: UNIVERSITY OF PITTSBURGH AT PITTSBURGH
• 依托单位国家: 美国
• 财政年份: 2006
• 资助国家: 美国
• 起止时间: 2006-07-12 至 2009-04-30
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/7129705
• 关键词: motivation; performance

DESCRIPTION (provided by applicant): Observer (and non-observer) performance studies are routinely performed to evaluate technologies and practices in medical imaging for possible use in the clinical environment as well as during the regulatory approval process. A large fraction of these evaluations to date, including but not limited to those performed in the field of medical imaging, required task specific ROC type response (ratings) for the analyses. Recent advancements have led to the development of analytical tools that enable the comparison of data ascertained under the FROC paradigm, which may have substantial advantages over the ROC paradigm for many of the applications of interest in this field. The FROC approach may also prove to more closely follow (emulate) the actual clinical diagnostic process, hence generalizability of and inferences based on study results could have more validity (relevance) than that attainable with other approaches, the overall objective of this project is to improve our understanding of the practical (methodological) and computational (analyses) related issues that one encounters during systems' performance studies when the experimental paradigm includes FROC type ratings and the observer, a computerized scheme (e.g. CAD), or a combination of both become an integral part of the diagnostic system. Building on a new approach to analyze scoring data ascertained during the performance of ROC type studies, we propose to develop non- parametric permutation based tests to compare FROC rating datasets in a paired design. The underlying approach compares observations between all pairs of actually negative and positive findings (e.g. cancer at a specific location) to construct summary indices of performance using a non-parametric methodology. Both unconditional and conditional (when appropriate) tests will be developed and tested. The proposed, including its advantages and limitations under different study conditions, will be investigated in a comprehensive manner using both extensive simulations as experimentally ascertained datasets.

描述（由申请人提供）：定期进行观察者（和非观察者）性能研究，以评估医学成像技术和实践，以便在临床环境以及监管审批过程中可能使用。迄今为止，大部分评估（包括但不限于在医学成像领域进行的评估）都需要特定任务的 ROC 类型响应（评级）进行分析。最近的进步导致了分析工具的发展，这些工具能够比较 FROC 范式下确定的数据，对于该领域的许多感兴趣的应用来说，这可能比 ROC 范式具有巨大的优势。 FROC 方法也可能被证明更紧密地遵循（模拟）实际的临床诊断过程，因此基于研究结果的普遍性和推论可能比其他方法具有更高的有效性（相关性），该项目的总体目标是当实验范式包括 FROC 类型评级和观察者、计算机化方案（例如 CAD）时，提高我们对系统性能研究期间遇到的实际（方法）和计算（分析）相关问题的理解，或两者的组合成为诊断系统的组成部分。基于分析 ROC 类型研究期间确定的评分数据的新方法，我们建议开发基于非参数排列的测试来比较配对设计中的 FROC 评级数据集。基本方法比较所有实际阴性和阳性结果（例如特定位置的癌症）之间的观察结果，以使用非参数方法构建绩效总结指数。将开发和测试无条件和有条件（适当时）测试。所提出的方案，包括其在不同研究条件下的优点和局限性，将使用广泛的模拟作为实验确定的数据集进行全面的研究。