mlsanityData to 14 Oct 2026

Calibrated and Conformal Propensity Scores for Causal Effect Estimation

arXiv 2306.00382 ↗First posted 1 Jun 2023stat.ME cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

Propensity scores are commonly used to estimate treatment effects from observational data. We argue that the probabilistic output of a learned propensity score model should be calibrated -- i.e., a predictive treatment probability of 90% should correspond to 90% of individuals being assigned the treatment group -- and we propose simple recalibration techniques to ensure this property. We prove that calibration is a necessary condition for unbiased treatment effect estimation when using popular inverse propensity weighted and doubly robust estimators. We derive error bounds on causal effect estimates that directly relate to the quality of uncertainties provided by the probabilistic propensity score model and show that calibration strictly improves this error bound while also avoiding extreme propensity weights. We demonstrate improved causal effect estimation with calibrated propensity scores in several tasks including high-dimensional image covariates and genome-wide association studies (GWASs). Calibrated propensity scores improve the speed of GWAS analysis by more than two-fold by enabling the use of simpler models that are faster to train.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 18 of its 61 references that are in the map. First posted on arXiv 1 Jun 2023.

Builds on

18 of 61 references.

References · 18

  1. 2017On Calibration of Modern Neural Networkscited by 4,960
  2. 2021A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantificationcited by 792
  3. 2018Accurate Uncertainties for Deep Learning Using Calibrated Regressioncited by 422
  4. 2017Causal Effect Inference with Deep Latent-Variable Modelscited by 410
  5. 2018The Blessings of Multiple Causescited by 104
  6. 2016Balancing Covariates via Propensity Score Weightingcited by 65
  7. 2018DeepMatch: Balancing Deep Covariate Representations for Causal Inference Using Adversarial Trainingcited by 42
  8. 2016Covariate Balancing Propensity Score by Tailored Loss Functionscited by 34
  9. 2021Calibrated and Sharp Uncertainties in Deep Learning via Density Estimationcited by 33
  10. 2019Calibrated Model-Based Deep Reinforcement Learningcited by 31
  11. 2017Estimation of causal effects with multiple treatments: a review and new ideascited by 29
  12. 2021The Balancing Act in Causal Inferencecited by 20
6 more
  1. 2023Causal isotonic calibration for heterogeneous treatment effectscited by 16
  2. 2017Regularized calibrated estimation of propensity scores with model misspecification and high-dimensional datacited by 15
  3. 2022Deep Multi-Modal Structural Equations For Causal Effect Estimation With Unstructured Proxiescited by 13
  4. 2018Robust Estimation of Causal Effects via High-Dimensional Covariate Balancing Propensity Scorecited by 12
  5. 2022Calibration Error for Heterogeneous Treatment Effectscited by 5
  6. 2020Tutorial: Introduction to computational causal inference using reproducible Stata, R and Python codecited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 0; 2026: 2.

3 papers.

  1. 2024Improving the Finite Sample Estimation of Average Treatment Effects using Double/Debiased Machine Learning with Propensity Score Calibrationcited by 5
  2. 2026Tailoring Strictly Proper Scoring Rules for Downstream Tasks: An Application to Causal Inferencecited by 1
  3. 2026Bellman Calibration for Marginalized Importance Weighting in Offline Reinforcement Learningcited by 0