Discuss the challenges and solutions of implementing propensity score matching in observational studies.

Official answer

Define the treatment, outcome, target population and estimand before matching. Estimate the propensity score—the probability of treatment given selected pretreatment covariates—using substantive knowledge about confounding. Exclude inappropriate post-treatment variables; a model with excellent treatment prediction is not necessarily a good causal design.

Check overlap. Treated units with no comparable controls cannot be repaired by forcing distant matches. Choose matching/caliper and replacement rules appropriate to the estimand, and report exclusions because they change the population represented.

Assess covariate balance directly, including standardized differences and relevant distributional features. Balance-test p-values are not a good pass/fail criterion: they depend strongly on sample size, and a nonsignificant test does not establish equality. Similar propensity scores alone do not guarantee every covariate is adequately balanced.

Estimate outcomes with uncertainty that respects matching, reused controls and the study design. Matching addresses measured covariates under assumptions such as conditional exchangeability, positivity and consistency; it does not reproduce randomization or eliminate unmeasured confounding. Report sensitivity to plausible unmeasured causes and explain limits of generalization.

Reference: MatchIt guidance on assessing balance.