An empirical assessment on the robustness of propensity score estimation to unobserved covariates

Propensity score analysis is one of the increasingly popular techniques to deal with selection bias in observational studies for causal inference. Unfortunately, propensity score estimation as a critical component of propensity score analysis to reduce selection bias can only account for observed covariates. Several strategies have been proposed to tackle this issue through sensitivity analysis, but the behavior of the robustness of propensity score estimation to unobserved covariates has not been fully understood and still needs further investigations. The present study introduces an empirical assessment on the robustness of propensity score estimation to unobserved covariates. The robustness is assessed through examining a probability that an estimated propensity score would cross a threshold if an unobserved covariate were included in the propensity score estimation model. To calculate the probability, a reference distribution of the estimated propensity score assuming that an unobserved covariate were in the model is derived by borrowing information from observed covariates, and then the reference distribution is utilized as a sampling distribution of the estimated propensity score assuming that an unobserved covariate were in the model. This technique of empirically assessing the robustness of propensity score estimation to unobserved covariates is applied to real data on substance abuse prevention for high-risk youth. The implications and limitations of the empirical assessment on the robustness of propensity score estimation to unobserved covariates are also discussed.