# Synth-Validation: Selecting the Best Causal Inference Method for a Given Dataset

arXiv: https://arxiv.org/abs/1711.00083 · HTML: https://mlsanity.com/paper/1711.00083/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Oct 2017.

## Built on by (most cited first, 9 of 9)

- [In Search of Insights, Not Magic Bullets: Towards Demystification of the Model Selection Dilemma in Heterogeneous Treatment Effect Estimation](https://mlsanity.com/paper/2302.02923/) (2023)
- [The Challenges of Hyperparameter Tuning for Accurate Causal Effect Estimation](https://mlsanity.com/paper/2303.01412/) (2023)
- [Ensemble Method for Estimating Individualized Treatment Effects](https://mlsanity.com/paper/2202.12445/) (2022)
- [Language Models as Causal Effect Generators](https://mlsanity.com/paper/2411.08019/) (2024)
- [Unveiling the Potential of Robustness in Selecting Conditional Average Treatment Effect Estimators](https://mlsanity.com/paper/2402.18392/) (2024)
- [DagSim: Combining DAG-based model structure with unconstrained data types and relations for flexible, transparent, and modularized data simulation](https://mlsanity.com/paper/2205.11234/) (2022)
- [Machine Learning Prescriptive Canvas for Optimizing Business Outcomes](https://mlsanity.com/paper/2206.10333/) (2022)
- [Consistent Labeling Across Group Assignments: Variance Reduction in Conditional Average Treatment Effect Estimation](https://mlsanity.com/paper/2507.04332/) (2025)
- [Improving Generative Methods for Causal Evaluation via Simulation-Based Inference](https://mlsanity.com/paper/2509.02892/) (2025)

## Abstract (arXiv)

Many decisions in healthcare, business, and other policy domains are made without the support of rigorous evidence due to the cost and complexity of performing randomized experiments. Using observational data to answer causal questions is risky: subjects who receive different treatments also differ in other ways that affect outcomes. Many causal inference methods have been developed to mitigate these biases. However, there is no way to know which method might produce the best estimate of a treatment effect in a given study. In analogy to cross-validation, which estimates the prediction error of predictive models applied to a given dataset, we propose synth-validation, a procedure that estimates the estimation error of causal inference methods applied to a given dataset. In synth-validation, we use the observed data to estimate generative distributions with known treatment effects. We apply each causal inference method to datasets sampled from these distributions and compare the effect estimates with the known effects to estimate error. Using simulations, we show that using synth-validation to select a causal inference method for each study lowers the expected estimation error relative to consistently using any single method.