Evaluating Agreement Among Observers or Methods of Measurement forQuantitative Data 公开
Wiener, Jeffrey Benjamin (2009)
Abstract
Evaluating Agreement Among Observers or Methods of Measurement for Quantitative Data By Jeffrey Wiener
Agreement measures are used to compare measurements of a specific variable made by different observers or methods, and to evaluate whether a substantial difference exists between these sources of measurement. Assessing agreement is applicable to method comparison or observer reliability studies in both the biomedical and psychosocial sciences. Frequently, a reference method or gold standard exists which is considered to be the most accurate of those available.
First, we explore and evaluate multiple unscaled measures of agreement between quantitative measurements by two observers with and without replications. Two scaled coefficients of agreement based on a general disagreement function which makes no distributional assumptions are described, one for the case of no applicable reference method, and the second for the case where one observer is considered a reference. We develop methods of inference for these coefficients, evaluating them against previously developed methods, and also define the asymptotic distribution of the coefficients and assess the robustness of the estimation methods. Next, we extend the described coefficients of agreement to the case where a set of two or more observers are selected at random from a pool of potential observers.
Finally, we model agreement using a disagreement function as our outcome variable. The effects of subject-specific covariates are examined. We apply these methods to a behavioral intervention study on medication adherence in HIV-positive children and to a carotid stenosis screening method comparison study.
Table of Contents
Table of Contents
1. Introduction...1
2. Approaches for Evaluating Agreement Between Two Observers...5
2.1 Introduction and notation...5 2.2 Existing methods...6
2.2.1 Mean Squared Deviation (MSD)...6 2.2.2 Intraclass Correlation Coefficient (ICC)...7 2.2.3 Concordance Correlation Coefficient (CCC)...10 2.2.4 Total Deviation Index (TDI)...11 2.2.5 Coverage Probability (CP)...12
2.3 Evaluation of existing methods...13 2.4 Discussion...15
3. A General Approach for Evaluating Agreement Between Two Observers with Replicated Measurements...22
3.1 Introduction and notation...22 3.2 Coefficients of Individual Agreement...23 3.3 Methods of inference for the Coefficients of Individual Agreement using the MSD...28
3.3.1 Method A - assuming independence between estimated mean square errors...28 3.3.2 Method B - general method using subject-specific estimates...30 3.3.3 Method C - estimation and inference using variance components...31
3.4 Simulation study - performance and comparison of estimates...34 3.5 Sample size estimation...36 3.6 Examples...37
3.6.1 Bland and Altman Systolic Blood Pressure (SBP) Data...37 3.6.2 Carotid Stenosis Data...38
3.7 Robustness of estimates and standard errors...38 3.8 Extension to J ≥ 2 observers...40 3.9 Discussion...42
4. A General Approach for Evaulating Agreement Between Observers Selected at Random...50
4.1 Introduction...50 4.2 Coefficients of Individual Agreement for random observers...51 4.3 Estimation and inference...54
4.3.1 Estimation and inference for ψN...54 4.3.2 Estimation and inference for ψR...55 4.3.3 Estimation and inference using variance components...57
4.4 Simulation study - performance and comparison of estimates...58 4.5 Examples...60
4.5.1 Bland and Altman Blood Pressure (SBP) Data...60 4.5.2 Carotid Stenosis Data...60
4.6 Discussion...61
5. Modeling Measures of Agreement 65 5.1 Introduction and notation...65
5.2 Pediatric Impact adherence data...66 5.3 Models...67
5.3.1 Two observers with covariates - least squares method...67 5.3.2 More than two observers, no reference - mixed model...68 5.3.3 More than two observers with a reference method - mixed model...69 5.3.4 Penalized spline regression models...70
5.4 Carotid stenosis data...72 5.5 Discussion...73
6. Summary...81 Bibliography...82
About this Dissertation
| School | |
|---|---|
| Department | |
| Degree | |
| Submission | |
| Language |
|
| Research Field | |
| 关键词 | |
| Committee Chair / Thesis Advisor | |
| Committee Members |
Primary PDF
| Thumbnail | Title | Date Uploaded | Actions |
|---|---|---|---|
|
|
Evaluating Agreement Among Observers or Methods of Measurement forQuantitative Data () | 2018-08-28 12:33:15 -0400 |
|
Supplemental Files
| Thumbnail | Title | Date Uploaded | Actions |
|---|