mlsanityData to 14 Oct 2026

In Search of Insights, Not Magic Bullets: Towards Demystification of the Model Selection Dilemma in Heterogeneous Treatment Effect Estimation

arXiv 2302.02923 ↗First posted 6 Feb 2023stat.ML cs.LG econ.EMcited by 29

Not on the map. Why?

Abstract (arXiv)

Personalized treatment effect estimates are often of interest in high-stakes applications -- thus, before deploying a model estimating such effects in practice, one needs to be sure that the best candidate from the ever-growing machine learning toolbox for this task was chosen. Unfortunately, due to the absence of counterfactual information in practice, it is usually not possible to rely on standard validation metrics for doing so, leading to a well-known model selection dilemma in the treatment effect estimation literature. While some solutions have recently been investigated, systematic understanding of the strengths and weaknesses of different model selection criteria is still lacking. In this paper, instead of attempting to declare a global `winner', we therefore empirically investigate success- and failure modes of different selection criteria. We highlight that there is a complex interplay between selection strategies, candidate estimators and the data used for comparing them, and provide interesting insights into the relative (dis)advantages of different criteria alongside desiderata for the design of further illuminating empirical studies in this context.

Key facts. Cited by 29 papers in the map (data through 14 Oct 2026). Builds on 21 of its 41 references that are in the map. First posted on arXiv 6 Feb 2023.

Builds on

21 of 41 references.

References · 21

  1. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
  2. 2015Estimation and Inference of Heterogeneous Treatment Effects using Random Forestscited by 600
  3. 2017Meta-learners for Estimating Heterogeneous Treatment Effects using Machine Learningcited by 478
  4. 2017Causal Effect Inference with Deep Latent-Variable Modelscited by 410
  5. 2017Quasi-Oracle Estimation of Heterogeneous Treatment Effectscited by 332
  6. 2016Learning Representations for Counterfactual Inferencecited by 329
  7. 2016Generalized Random Forestscited by 312
  8. 2015Recursive Partitioning for Heterogeneous Causal Effectscited by 255
  9. 2021Nonparametric Estimation of Heterogeneous Treatment Effects: From Theory to Learning Algorithmscited by 144
  10. 2017Automated versus do-it-yourself methods for causal inference: Lessons learned from a data analysis competitioncited by 106
  11. 2021Discussion on "Bayesian Regression Tree Models for Causal Inference: Regularization, Confounding, and Heterogeneous Effects" by Hahn, Murray and Carvalhocited by 77
  12. 2021On Inductive Biases for Heterogeneous Treatment Effect Estimationcited by 75
9 more
  1. 2020Counterfactual Representation Learning with Balancing Weightscited by 45
  2. 2018Learning Weighted Representations for Generalization Across Designscited by 41
  3. 2019Using Wasserstein Generative Adversarial Networks for the Design of Monte Carlo Simulationscited by 32
  4. 2018A comparison of methods for model selection when estimating individual treatment effectscited by 28
  5. 2022Empirical Analysis of Model Selection for Heterogeneous Causal Effect Estimationcited by 25
  6. 2022Validating Causal Inference Methodscited by 20
  7. 2017Synth-Validation: Selecting the Best Causal Inference Method for a Given Datasetcited by 9
  8. 2019Selective machine learning of doubly robust functionalscited by 9
  9. 2021β-Intact-VAE: Identifying and Estimating Causal Effects under Limited Overlapcited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 8; 2024: 10; 2025: 8; 2026: 3.

29 papers.

Sort:

By area:

  1. 2024Causal machine learning for predicting treatment outcomescited by 74
  2. 2024DiffPO: A causal diffusion model for learning distributions of potential outcomescited by 17
  3. 2023Dynamic Inter-treatment Information Sharing for Individualized Treatment Effects Estimationcited by 11
  4. 2024Quantifying Aleatoric Uncertainty of the Treatment Effect: A Novel Orthogonal Learnercited by 10
  5. 2023The Challenges of Hyperparameter Tuning for Accurate Causal Effect Estimationcited by 6
  6. 2025Position: Causal Machine Learning Requires Rigorous Synthetic Experiments for Broader Adoptioncited by 6
  7. 2025Orthogonal Representation Learning for Estimating Causal Quantitiescited by 5
  8. 2023Causal Q-Aggregation for CATE Model Selectioncited by 4
  9. 2023Robustness of Algorithms for Causal Structure Learning to Hyperparameter Choicecited by 4
  10. 2024Optimizing Treatment Allocation in the Presence of Interferencecited by 4
  11. 2025Improving the Generation and Evaluation of Synthetic Data for Downstream Medical Causal Inferencecited by 4
  12. 2023Transfer Learning for Causal Effect Estimationcited by 3
17 more
  1. 2023Learned Causal Method Predictioncited by 3
  2. 2024PairNet: Training with Observed Pairs to Estimate Individual Treatment Effectcited by 3
  3. 2024Federated Learning for Estimating Heterogeneous Treatment Effectscited by 2
  4. 2024Unveiling the Potential of Robustness in Selecting Conditional Average Treatment Effect Estimatorscited by 2
  5. 2024Defining Expertise: Applications to Treatment Effect Estimationcited by 2
  6. 2024Sources of Gain: Decomposing Performance in Conditional Average Dose Response Estimationcited by 2
  7. 2023Assessing Electricity Service Unfairness with Transfer Counterfactual Learningcited by 1
  8. 2025Overlap-Adaptive Regularization for Conditional Average Treatment Effect Estimationcited by 1
  9. 2025A Relative Error-Based Evaluation Framework of Heterogeneous Treatment Effect Estimatorscited by 1
  10. 2025Estimating Treatment Effects in Networks under Unknown Exposure Mappingscited by 1
  11. 2026Evaluating Uplift Modeling under Structural Biases: Insights into Metric Stability and Model Robustnesscited by 1
  12. 2023Using representation balancing to learn conditional-average dose responses from clustered datacited by 0
  13. 2024Robust CATE Estimation Using Novel Ensemble Methodscited by 0
  14. 2025Heterogeneous Treatment Effect in Time-to-Event Outcomes: Harnessing Censored Data with Recursively Imputed Treescited by 0
  15. 2025Leveraging a Simulator for Learning Causal Representations from Post-Treatment Covariates for CATEcited by 0
  16. 2026Bayesian X-Learner: Calibrated Posterior Inference for Heterogeneous Treatment Effects under Heavy-Tailed Outcomescited by 0
  17. 2026Real vs. Semi-Simulated: Rethinking Evaluation for Treatment Effect Estimationcited by 0