# Propensity score models are better when post-calibrated

arXiv: https://arxiv.org/abs/2211.01221 · HTML: https://mlsanity.com/paper/2211.01221/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 44 references that are in the map.
- First posted on arXiv 2 Nov 2022.

## Builds on (most cited first, 3 of 3)

- [Automated versus do-it-yourself methods for causal inference: Lessons learned from a data analysis competition](https://mlsanity.com/paper/1707.02641/) (2017)
- [An Evaluation Toolkit to Guide Model Selection and Cohort Definition in Causal Inference](https://mlsanity.com/paper/1906.00442/) (2019)
- [To tune or not to tune, a case study of ridge logistic regression in small or sparse datasets](https://mlsanity.com/paper/2101.11230/) (2021)

## Built on by (most cited first, 3 of 3)

- [Calibeating Prediction-Powered Inference](https://mlsanity.com/paper/2604.21260/) (2026)
- [Hierarchical Bias-Driven Stratification for Interpretable Causal Effect Estimation](https://mlsanity.com/paper/2401.17737/) (2024)
- [From Uncertainty to Precision: Enhancing Binary Classifier Performance through Calibration](https://mlsanity.com/paper/2402.07790/) (2024)

## Abstract (arXiv)

Theoretical guarantees for causal inference using propensity scores are partly based on the scores behaving like conditional probabilities. However, scores between zero and one, especially when outputted by flexible statistical estimators, do not necessarily behave like probabilities. We perform a simulation study to assess the error in estimating the average treatment effect before and after applying a simple and well-established post-processing method to calibrate the propensity scores. We find that post-calibration reduces the error in effect estimation for expressive uncalibrated statistical estimators, and that this improvement is not mediated by better balancing. The larger the initial lack of calibration, the larger the improvement in effect estimation, with the effect on already-calibrated estimators being very small. Given the improvement in effect estimation and that post-calibration is computationally cheap, we recommend it will be adopted when modelling propensity scores with expressive models.