New Methods for Evaluating Interobserver Agreement

评估观察者间一致性的新方法

基本信息

批准号：
7103652
负责人：
Michael J Haber
金额：
$ 24.3万
依托单位：
EMORY UNIVERSITY
依托单位国家：
美国
项目类别：
财政年份：
2004
资助国家：
美国
起止时间：
2004-07-01 至 2007-12-30
项目状态：
已结题

来源：
https://reporter.nih.gov/project-details/7103652
关键词：
AIDS education /prevention artery stenosis behavior test behavioral /social science research tag blood pressure clinical research computer program /software computer simulation data collection methodology /evaluation diet therapy human data method development statistics /biometry

项目摘要

DESCRIPTION (provided by applicant): Behavioral, social and biomedical researchers use a variety of instruments to measure a wide range of personal, environmental, behavioral and biomedical variables. Before a new measurement scale is adopted for practical use, one needs to evaluate inter, intra and total agreement among various observers and/or measurement methods. In this proposal we take a fresh look into the field of assessing observer agreement for continuous measurement scales. Existing indices of agreement, such as the intra-class correlation coefficient, indices based on generalizability theory (GT) and the concordance correlation coefficient measure the total observer agreement, which combines the inter and intra observer components of agreement. Currently there is no proper inter-observer agreement index for assessing the "true" agreement among observers in terms of true (i.e., error-free) values of different observers on the same subject. The first major objective of this project is to develop two new indices of interobserver agreement defined in terms of the distances among the true values assigned to each subject by several observers. In the proposed research, we will develop methods for statistical inference on the new interobserver agreement indices and investigate their properties via simulations. The second major objective of this project is to develop and investigate innovative methods for statistical inference on the GT-based indices. In GT it is usually assumed that the observations follow an analysis of variance (ANOVA) model and inference is made under the normality assumption. These assumptions are often unrealistic. We will extend GT concepts to more general settings and develop nonparametric, Semi-parametric and parametric methods for estimation and inference. Existing and new agreement indices will be compared and linked. All the proposed methods will be applied to real data sets from the behavioral and biomedical sciences.

描述（由申请人提供）：行为，社会和生物医学研究人员使用各种工具来衡量广泛的个人，环境，行为和生物医学变量。在采用新的测量量表进行实际使用之前，需要评估各种观察者和/或测量方法之间的间，内部和总共识。在此提案中，我们将重新查看评估持续测量量表的观察者一致性领域。现有的一致性指标，例如基于类内部相关系数，基于概括性理论（GT）的指标以及一致性相关系数衡量总观察者一致性，该协议结合了一致性和内部观察者的一致性。当前，没有适当的观察者协议指数来评估观察者之间在同一主题上不同观察者的真实（即无错误）值的“真实”协议。该项目的第一个主要目的是根据几个观察者分配给每个主题的真实值之间的距离，开发两个新的观察者一致性指数。在拟议的研究中，我们将开发出对新的观察者一致性指数统计推断的方法，并通过模拟调查其属性。该项目的第二个主要目标是开发和研究基于GT指数的统计推断的创新方法。在GT中，通常假定观测值遵循方差分析（ANOVA）模型，并且在正态性假设下进行推断。这些假设通常是不现实的。我们将将GT概念扩展到更一般的设置，并开发非参数，半参数和参数方法进行估计和推理。现有和新协议指数将被比较并链接。所有提出的方法将应用于行为和生物医学科学的实际数据集。