Impact of Predictive Ability on Identifying Higher-Risk Population for Common Diseases in Polygenic Risk Prediction Open Access

Abstract

Objective: Polygenic risk scores can identify individuals with increased risk for common diseases. However, the magnitude of increased risk is unclear for different predictive ability, which is assessed by the area under the receiver operating characteristic curve (AUC). Sensitivity, specificity, PPV, NPV and reclassification measures can reflect predictive performance from various aspects. We aimed to investigate how the magnitude of increased risk for higher-risk group and the above parameters for predictive performance will vary by increasing the AUC.

Methods: We simulated hypothetical risk data with genetic variants and disease status for 100,000 individuals and constructed risk prediction models with the AUC ranging from 0.60 to 0.80. Predicted risks were calculated from Bayes’ theorem and logistic regression. We first replicated the findings in the study of Khera et al., then examined odds ratio (OR) for higher-risk group, sensitivity, specificity, PPV and NPV when AUC improved against different cut-offs that can define higher-risk group. We also explored the relationship between reclassification measures (IDI, percentage of total reclassification, NRI and reclassification improvement of cases and non-cases) and increment of AUC (∆AUC).

Results: OR of the higher-risk population (versus the remainder) increased with improving AUC at an increasing rate for a fixed risk threshold; sensitivity and PPV increased with increasing AUC, but specificity and NPV remained almost constant when AUC improved. IDI, percentage of total reclassification and NRI increased with increasing ∆AUC; the reclassification improvement was higher for cases than for non-cases at the same ∆AUC.

Conclusions: We can identify the higher-risk population with increased OR for common diseases across risk thresholds compared with the remainder when the predictive ability of genetic risk model improves. The sensitivity and PPV increase with improving AUC, and this influence varies across different risk thresholds. Reclassification measures favorably increase when the ∆AUC improves, which is achieved mainly by improving the reclassification of individuals with events.

Table of Contents

Table of Contents

Introduction .......................................................................................................................................... 1

Method .................................................................................................................................................. 4

Data simulation .......................................................................................................................................... 4

Statistical analyses ..................................................................................................................................... 6

Results ................................................................................................................................................... 9

Validation of the Odds ratio for high-risk group in the study of Khera et al. ............................................. 9

Investigate how OR of the higher-risk group varied with the predictive ability ....................................... 12

Investigate how predictive performance varied with the predictive ability .............................................. 14

Investigate how reclassification measures varied with the predictive ability ........................................... 16

Correct and incorrect moves for cases and non-cases ............................................................................. 20

Discussion ............................................................................................................................................ 21

Strengths and Weaknesses ................................................................................................................... 26

Conclusions ......................................................................................................................................... 27

References ........................................................................................................................................... 28

Appendix 1: Definitions and Formulas for Reclassification Measures ................................................ 30

Appendix 2: Comparison of the Validation Results and the Study of Khera et al. ............................. 32

Appendix 3: Risk Distributions of Cases and Non-cases for Varied AUCs ......................................... 33

About this Master's Thesis

Rights statement

• Permission granted by the author to include this thesis or dissertation in this repository. All rights reserved by the author. Please contact the author for information regarding the reproduction and use of this thesis or dissertation.

School	Rollins School of Public Health
Department	Epidemiology
Subfield / Discipline	Epidemiology - MPH & MSPH
Degree	M.S.P.H.
Submission	Master's Thesis
Language	English
Research Field	Biology, Genetics Health Sciences, Epidemiology Health Sciences, Public Health
Keyword	Area under curve; Polygenic risk; Reclassification; Prediction
Committee Chair / Thesis Advisor	Janssens, A. Cecile J.W., Emory University

Last modified

Primary PDF

Thumbnail	Title	Date Uploaded	Actions
	Impact of Predictive Ability on Identifying Higher-Risk Population for Common Diseases in Polygenic Risk Prediction ()	2019-04-24 13:39:44 -0400	Press to Select an action Download

Supplemental Files

Thumbnail	Title	Date Uploaded	Actions