# Calibrated and Conformal Propensity Scores for Causal Effect Estimation

arXiv: https://arxiv.org/abs/2306.00382 · HTML: https://mlsanity.com/paper/2306.00382/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 61 references that are in the map.
- First posted on arXiv 1 Jun 2023.

## Builds on (most cited first, 18 of 18)

- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification](https://mlsanity.com/paper/2107.07511/) (2021)
- [Accurate Uncertainties for Deep Learning Using Calibrated Regression](https://mlsanity.com/paper/1807.00263/) (2018)
- [Causal Effect Inference with Deep Latent-Variable Models](https://mlsanity.com/paper/1705.08821/) (2017)
- [The Blessings of Multiple Causes](https://mlsanity.com/paper/1805.06826/) (2018)
- [Balancing Covariates via Propensity Score Weighting](https://mlsanity.com/paper/1609.07494/) (2016)
- [DeepMatch: Balancing Deep Covariate Representations for Causal Inference Using Adversarial Training](https://mlsanity.com/paper/1802.05664/) (2018)
- [Covariate Balancing Propensity Score by Tailored Loss Functions](https://mlsanity.com/paper/1601.05890/) (2016)
- [Calibrated and Sharp Uncertainties in Deep Learning via Density Estimation](https://mlsanity.com/paper/2112.07184/) (2021)
- [Calibrated Model-Based Deep Reinforcement Learning](https://mlsanity.com/paper/1906.08312/) (2019)
- [Estimation of causal effects with multiple treatments: a review and new ideas](https://mlsanity.com/paper/1701.05132/) (2017)
- [The Balancing Act in Causal Inference](https://mlsanity.com/paper/2110.14831/) (2021)
- [Causal isotonic calibration for heterogeneous treatment effects](https://mlsanity.com/paper/2302.14011/) (2023)
- [Regularized calibrated estimation of propensity scores with model misspecification and high-dimensional data](https://mlsanity.com/paper/1710.08074/) (2017)
- [Deep Multi-Modal Structural Equations For Causal Effect Estimation With Unstructured Proxies](https://mlsanity.com/paper/2203.09672/) (2022)
- [Robust Estimation of Causal Effects via High-Dimensional Covariate Balancing Propensity Score](https://mlsanity.com/paper/1812.08683/) (2018)
- [Calibration Error for Heterogeneous Treatment Effects](https://mlsanity.com/paper/2203.13364/) (2022)
- [Tutorial: Introduction to computational causal inference using reproducible Stata, R and Python code](https://mlsanity.com/paper/2012.09920/) (2020)

## Built on by (most cited first, 3 of 3)

- [Improving the Finite Sample Estimation of Average Treatment Effects using Double/Debiased Machine Learning with Propensity Score Calibration](https://mlsanity.com/paper/2409.04874/) (2024)
- [Tailoring Strictly Proper Scoring Rules for Downstream Tasks: An Application to Causal Inference](https://mlsanity.com/paper/2606.03332/) (2026)
- [Bellman Calibration for Marginalized Importance Weighting in Offline Reinforcement Learning](https://mlsanity.com/paper/2608.24858/) (2026)

## Abstract (arXiv)

Propensity scores are commonly used to estimate treatment effects from observational data. We argue that the probabilistic output of a learned propensity score model should be calibrated -- i.e., a predictive treatment probability of 90% should correspond to 90% of individuals being assigned the treatment group -- and we propose simple recalibration techniques to ensure this property. We prove that calibration is a necessary condition for unbiased treatment effect estimation when using popular inverse propensity weighted and doubly robust estimators. We derive error bounds on causal effect estimates that directly relate to the quality of uncertainties provided by the probabilistic propensity score model and show that calibration strictly improves this error bound while also avoiding extreme propensity weights. We demonstrate improved causal effect estimation with calibrated propensity scores in several tasks including high-dimensional image covariates and genome-wide association studies (GWASs). Calibrated propensity scores improve the speed of GWAS analysis by more than two-fold by enabling the use of simpler models that are faster to train.