mlsanityData to 7 Oct 2026

Empirical Analysis of Model Selection for Heterogeneous Causal Effect Estimation

arXiv 2211.01939 ↗First posted 3 Nov 2022cs.LG cs.AI stat.MEcited by 25

Not on the map. Why?

Abstract (arXiv)

We study the problem of model selection in causal inference, specifically for conditional average treatment effect (CATE) estimation. Unlike machine learning, there is no perfect analogue of cross-validation for model selection as we do not observe the counterfactual potential outcomes. Towards this, a variety of surrogate metrics have been proposed for CATE model selection that use only observed data. However, we do not have a good understanding regarding their effectiveness due to limited comparisons in prior studies. We conduct an extensive empirical analysis to benchmark the surrogate model selection metrics introduced in the literature, as well as the novel ones introduced in this work. We ensure a fair comparison by tuning the hyperparameters associated with these metrics via AutoML, and provide more detailed trends by incorporating realistic datasets via generative modeling. Our analysis suggests novel model selection strategies based on careful hyperparameter selection of CATE estimators and causal ensembling.

Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). Builds on 21 of its 48 references that are in the map. First posted on arXiv 3 Nov 2022.

Builds on

21 of 48 references.

References · 21

  1. 2015Estimation and Inference of Heterogeneous Treatment Effects using Random Forestscited by 597
  2. 2017Meta-learners for Estimating Heterogeneous Treatment Effects using Machine Learningcited by 475
  3. 2017Causal Effect Inference with Deep Latent-Variable Modelscited by 408
  4. 2017Quasi-Oracle Estimation of Heterogeneous Treatment Effectscited by 328
  5. 2016Data-Efficient Off-Policy Policy Evaluation for Reinforcement Learningcited by 285
  6. 2019Adapting Neural Networks for the Estimation of Treatment Effectscited by 285
  7. 2018Neural Autoregressive Flowscited by 207
  8. 2019Orthogonal Statistical Learningcited by 150
  9. 2021Nonparametric Estimation of Heterogeneous Treatment Effects: From Theory to Learning Algorithmscited by 144
  10. 2016Optimal and Adaptive Off-policy Evaluation in Contextual Banditscited by 111
  11. 2017Automated versus do-it-yourself methods for causal inference: Lessons learned from a data analysis competitioncited by 106
  12. 2019FLAML: A Fast and Lightweight AutoML Librarycited by 104
9 more
  1. 2019Estimating Treatment Effects with Causal Forests: An Applicationcited by 58
  2. 2017The Relative Performance of Ensemble Methods with Deep Convolutional Neural Networks for Image Classificationcited by 50
  3. 2020RealCause: Realistic Causal Inference Benchmarkingcited by 44
  4. 2016Double/Debiased Machine Learning for Treatment and Causal Parameterscited by 41
  5. 2017Uplift Modeling with Multiple Treatments and General Response Typescited by 41
  6. 2018Benchmarking Framework for Performance-Evaluation of Causal Inference Analysiscited by 33
  7. 2018A comparison of methods for model selection when estimating individual treatment effectscited by 28
  8. 2020Stable discovery of interpretable subgroups via calibration in causal studiescited by 18
  9. 2022Ensemble Method for Estimating Individualized Treatment Effectscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 7; 2024: 4; 2025: 8; 2026: 6.

25 papers.

Sort:

By area:

  1. 2025CausalPFN: Amortized Causal Effect Estimation via In-Context Learningcited by 43
  2. 2023In Search of Insights, Not Magic Bullets: Towards Demystification of the Model Selection Dilemma in Heterogeneous Treatment Effect Estimationcited by 29
  3. 2023Towards Causal Foundation Model: on Duality between Causal Inference and Attentioncited by 17
  4. 2024DiffPO: A causal diffusion model for learning distributions of potential outcomescited by 17
  5. 2023Copula-Based Deep Survival Models for Dependent Censoringcited by 12
  6. 2023The Challenges of Hyperparameter Tuning for Accurate Causal Effect Estimationcited by 6
  7. 2025Position: Causal Machine Learning Requires Rigorous Synthetic Experiments for Broader Adoptioncited by 6
  8. 2023Causal Q-Aggregation for CATE Model Selectioncited by 4
  9. 2025Hybrid Meta-learners for Estimating Heterogeneous Treatment Effectscited by 4
  10. 2023Learned Causal Method Predictioncited by 3
  11. 2024Unveiling the Potential of Robustness in Selecting Conditional Average Treatment Effect Estimatorscited by 2
  12. 2024DAG-aware Transformer for Causal Effect Estimationcited by 2
13 more
  1. 2025RIDE: Enhancing Large Language Model Alignment through Restyled In-Context Learning Demonstration Exemplarscited by 2
  2. 2023Assessing Electricity Service Unfairness with Transfer Counterfactual Learningcited by 1
  3. 2025A Relative Error-Based Evaluation Framework of Heterogeneous Treatment Effect Estimatorscited by 1
  4. 2026Evaluating Uplift Modeling under Structural Biases: Insights into Metric Stability and Model Robustnesscited by 1
  5. 2026UpliftBench: Revealing Outcome-Regime and Objective Mismatch in Uplift Evaluationcited by 1
  6. 2024Robust CATE Estimation Using Novel Ensemble Methodscited by 0
  7. 2025A Generative Framework for Causal Estimation via Importance-Weighted Diffusion Distillationcited by 0
  8. 2025Shrinkage-Based Regressions with Many Related Treatmentscited by 0
  9. 2025Improving Generative Methods for Causal Evaluation via Simulation-Based Inferencecited by 0
  10. 2026Controllable Generative Sandbox for Causal Inferencecited by 0
  11. 2026InferenceEvolve: Towards Automated Causal Effect Estimators through Self-Evolving AIcited by 0
  12. 2026Real vs. Semi-Simulated: Rethinking Evaluation for Treatment Effect Estimationcited by 0
  13. 2026Causal Foundation Modelscited by 0