# Empirical Analysis of Model Selection for Heterogeneous Causal Effect Estimation

arXiv: https://arxiv.org/abs/2211.01939 · HTML: https://mlsanity.com/paper/2211.01939/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 48 references that are in the map.
- First posted on arXiv 3 Nov 2022.

## Builds on (most cited first, 21 of 21)

- [Estimation and Inference of Heterogeneous Treatment Effects using Random Forests](https://mlsanity.com/paper/1510.04342/) (2015)
- [Meta-learners for Estimating Heterogeneous Treatment Effects using Machine Learning](https://mlsanity.com/paper/1706.03461/) (2017)
- [Causal Effect Inference with Deep Latent-Variable Models](https://mlsanity.com/paper/1705.08821/) (2017)
- [Quasi-Oracle Estimation of Heterogeneous Treatment Effects](https://mlsanity.com/paper/1712.04912/) (2017)
- [Adapting Neural Networks for the Estimation of Treatment Effects](https://mlsanity.com/paper/1906.02120/) (2019)
- [Data-Efficient Off-Policy Policy Evaluation for Reinforcement Learning](https://mlsanity.com/paper/1604.00923/) (2016)
- [Neural Autoregressive Flows](https://mlsanity.com/paper/1804.00779/) (2018)
- [Orthogonal Statistical Learning](https://mlsanity.com/paper/1901.09036/) (2019)
- [Nonparametric Estimation of Heterogeneous Treatment Effects: From Theory to Learning Algorithms](https://mlsanity.com/paper/2101.10943/) (2021)
- [Optimal and Adaptive Off-policy Evaluation in Contextual Bandits](https://mlsanity.com/paper/1612.01205/) (2016)
- [Automated versus do-it-yourself methods for causal inference: Lessons learned from a data analysis competition](https://mlsanity.com/paper/1707.02641/) (2017)
- [FLAML: A Fast and Lightweight AutoML Library](https://mlsanity.com/paper/1911.04706/) (2019)
- [Estimating Treatment Effects with Causal Forests: An Application](https://mlsanity.com/paper/1902.07409/) (2019)
- [The Relative Performance of Ensemble Methods with Deep Convolutional Neural Networks for Image Classification](https://mlsanity.com/paper/1704.01664/) (2017)
- [RealCause: Realistic Causal Inference Benchmarking](https://mlsanity.com/paper/2011.15007/) (2020)
- [Double/Debiased Machine Learning for Treatment and Causal Parameters](https://mlsanity.com/paper/1608.00060/) (2016)
- [Uplift Modeling with Multiple Treatments and General Response Types](https://mlsanity.com/paper/1705.08492/) (2017)
- [Benchmarking Framework for Performance-Evaluation of Causal Inference Analysis](https://mlsanity.com/paper/1802.05046/) (2018)
- [A comparison of methods for model selection when estimating individual treatment effects](https://mlsanity.com/paper/1804.05146/) (2018)
- [Stable discovery of interpretable subgroups via calibration in causal studies](https://mlsanity.com/paper/2008.10109/) (2020)
- [Ensemble Method for Estimating Individualized Treatment Effects](https://mlsanity.com/paper/2202.12445/) (2022)

## Built on by (most cited first, 25 of 25)

- [CausalPFN: Amortized Causal Effect Estimation via In-Context Learning](https://mlsanity.com/paper/2506.07918/) (2025)
- [In Search of Insights, Not Magic Bullets: Towards Demystification of the Model Selection Dilemma in Heterogeneous Treatment Effect Estimation](https://mlsanity.com/paper/2302.02923/) (2023)
- [Towards Causal Foundation Model: on Duality between Causal Inference and Attention](https://mlsanity.com/paper/2310.00809/) (2023)
- [DiffPO: A causal diffusion model for learning distributions of potential outcomes](https://mlsanity.com/paper/2410.08924/) (2024)
- [Copula-Based Deep Survival Models for Dependent Censoring](https://mlsanity.com/paper/2306.11912/) (2023)
- [The Challenges of Hyperparameter Tuning for Accurate Causal Effect Estimation](https://mlsanity.com/paper/2303.01412/) (2023)
- [Position: Causal Machine Learning Requires Rigorous Synthetic Experiments for Broader Adoption](https://mlsanity.com/paper/2508.08883/) (2025)
- [Causal Q-Aggregation for CATE Model Selection](https://mlsanity.com/paper/2310.16945/) (2023)
- [Hybrid Meta-learners for Estimating Heterogeneous Treatment Effects](https://mlsanity.com/paper/2506.13680/) (2025)
- [Learned Causal Method Prediction](https://mlsanity.com/paper/2311.03989/) (2023)
- [Unveiling the Potential of Robustness in Selecting Conditional Average Treatment Effect Estimators](https://mlsanity.com/paper/2402.18392/) (2024)
- [DAG-aware Transformer for Causal Effect Estimation](https://mlsanity.com/paper/2410.10044/) (2024)
- [RIDE: Enhancing Large Language Model Alignment through Restyled In-Context Learning Demonstration Exemplars](https://mlsanity.com/paper/2502.11681/) (2025)
- [Assessing Electricity Service Unfairness with Transfer Counterfactual Learning](https://mlsanity.com/paper/2310.03258/) (2023)
- [A Relative Error-Based Evaluation Framework of Heterogeneous Treatment Effect Estimators](https://mlsanity.com/paper/2510.16419/) (2025)
- [Evaluating Uplift Modeling under Structural Biases: Insights into Metric Stability and Model Robustness](https://mlsanity.com/paper/2603.20775/) (2026)
- [UpliftBench: Revealing Outcome-Regime and Objective Mismatch in Uplift Evaluation](https://mlsanity.com/paper/2608.00915/) (2026)
- [Causal Foundation Models](https://mlsanity.com/paper/2609.03003/) (2026)
- [Robust CATE Estimation Using Novel Ensemble Methods](https://mlsanity.com/paper/2407.03690/) (2024)
- [A Generative Framework for Causal Estimation via Importance-Weighted Diffusion Distillation](https://mlsanity.com/paper/2505.11444/) (2025)
- [Shrinkage-Based Regressions with Many Related Treatments](https://mlsanity.com/paper/2507.01202/) (2025)
- [Improving Generative Methods for Causal Evaluation via Simulation-Based Inference](https://mlsanity.com/paper/2509.02892/) (2025)
- [Controllable Generative Sandbox for Causal Inference](https://mlsanity.com/paper/2603.03587/) (2026)
- [InferenceEvolve: Towards Automated Causal Effect Estimators through Self-Evolving AI](https://mlsanity.com/paper/2604.04274/) (2026)
- [Real vs. Semi-Simulated: Rethinking Evaluation for Treatment Effect Estimation](https://mlsanity.com/paper/2605.10430/) (2026)

## Abstract (arXiv)

We study the problem of model selection in causal inference, specifically for conditional average treatment effect (CATE) estimation. Unlike machine learning, there is no perfect analogue of cross-validation for model selection as we do not observe the counterfactual potential outcomes. Towards this, a variety of surrogate metrics have been proposed for CATE model selection that use only observed data. However, we do not have a good understanding regarding their effectiveness due to limited comparisons in prior studies. We conduct an extensive empirical analysis to benchmark the surrogate model selection metrics introduced in the literature, as well as the novel ones introduced in this work. We ensure a fair comparison by tuning the hyperparameters associated with these metrics via AutoML, and provide more detailed trends by incorporating realistic datasets via generative modeling. Our analysis suggests novel model selection strategies based on careful hyperparameter selection of CATE estimators and causal ensembling.