The Confounders Imbalance vs. Choices between Multiple Regression and Propensity Score Approaches Open Access

Gao, Xingyu (Spring 2021)

Permanent URL:


Background: The propensity score methods are widely used in observational studies as a tool for covariates balancing, especially for potential confounders. The multiple regression method and PS methods agree with each other when the baseline covariate balance is good. However, there is no clear guidance on deciding the degree of the baseline covariate balance and which method to adopt for analysis.

Methods and Materials: In this project, we created two series of simulation studies to examine the performance of PS matching with 0.2 and 0.1 calipers, ATE, ATM, and ATO PS weighting, and multiple regression under different levels of baseline covariate overlap between the two comparison groups. To create a relatively fair condition of comparison, we added two types of model misspecification to the outcome model. Instead of assessing the overlap of all covariates, we used propensity score as a summary of information. Specifically, in the simulation study, we used the overlapping coefficient (OVL) as a measurement of the degree of overlap propensity score distributions between the treatment and the control group. We evaluated the performance of different methods by absolute bias, MSE, and maximum standardized difference among all covariates related to the outcome.

Results: In the scenario that an interaction term was added in the outcome model, regardless of the strength of the interaction term or the level of model misspecification, when the OVL was above 77.0%, all methods agreed with each other. When the OVL is between 77% and 62%, ATE performed best among all methods. When OVL is below 62%, PS matching with caliper methods performed the best among all. A smaller caliper only helped to improve the matching quality when the model is almost correctly specified. ATM and ATO performed stably regardless of the OVL and the strength of model misspecification. ATO could achieve exact balance regardless of the strength of model misspecification and OVL.

Conclusion: In this paper, we propose using the OVL as a measurement of covariate balance before choosing analytic methods. When the OVL is good, multiple regression outperforms PS methods, and multiple methods can be used for cross-validation purposes. However, when the OVL is small, we proved that PS methods outperform multiple regression based on the simulation result. 

Table of Contents

1.   Introduction


1.2Propensity score

1.3 PS matching

1.4 PS weighting

1.5 Common support and overlapping coefficient

1.6 Standardized difference

2.   Method

2.1 Data generating process

2.2 Analysis process

3.   Results

4.   Discussion

5.   Conclusion



About this Master's Thesis

Rights statement
  • Permission granted by the author to include this thesis or dissertation in this repository. All rights reserved by the author. Please contact the author for information regarding the reproduction and use of this thesis or dissertation.
Subfield / Discipline
  • English
Research Field
Committee Chair / Thesis Advisor
Committee Members
Last modified

Primary PDF

Supplemental Files