# Validating Causal Inference Methods

arXiv: https://arxiv.org/abs/2202.04208 · HTML: https://mlsanity.com/paper/2202.04208/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 51 references that are in the map.
- First posted on arXiv 9 Feb 2022.

## Builds on (most cited first, 15 of 15)

- [An Introduction to Variational Autoencoders](https://mlsanity.com/paper/1906.02691/) (2019)
- [Estimation and Inference of Heterogeneous Treatment Effects using Random Forests](https://mlsanity.com/paper/1510.04342/) (2015)
- [Meta-learners for Estimating Heterogeneous Treatment Effects using Machine Learning](https://mlsanity.com/paper/1706.03461/) (2017)
- [Generalized Random Forests](https://mlsanity.com/paper/1610.01271/) (2016)
- [Grammar Variational Autoencoder](https://mlsanity.com/paper/1703.01925/) (2017)
- [Machine Learning Methods Economists Should Know About](https://mlsanity.com/paper/1903.10075/) (2019)
- [RealCause: Realistic Causal Inference Benchmarking](https://mlsanity.com/paper/2011.15007/) (2020)
- [The State of Applied Econometrics - Causality and Policy Evaluation](https://mlsanity.com/paper/1607.00699/) (2016)
- [Using Wasserstein Generative Adversarial Networks for the Design of Monte Carlo Simulations](https://mlsanity.com/paper/1909.02210/) (2019)
- [The Case for Evaluating Causal Models Using Interventional Measures and Empirical Data](https://mlsanity.com/paper/1910.05387/) (2019)
- [FLAME: A Fast Large-scale Almost Matching Exactly Approach to Causal Inference](https://mlsanity.com/paper/1707.06315/) (2017)
- [A Selective Review of Negative Control Methods in Epidemiology](https://mlsanity.com/paper/2009.05641/) (2020)
- [MALTS: Matching After Learning to Stretch](https://mlsanity.com/paper/1811.07415/) (2018)
- [How and Why to Use Experimental Data to Evaluate Methods for Observational Causal Inference](https://mlsanity.com/paper/2010.03051/) (2020)
- [Selective machine learning of doubly robust functionals](https://mlsanity.com/paper/1911.02029/) (2019)

## Built on by (most cited first, 20 of 20)

- [Can You Rely on Your Model Evaluation? Improving Model Evaluation with Synthetic Test Data](https://mlsanity.com/paper/2310.16524/) (2023)
- [CausalBench: A Large-scale Benchmark for Network Inference from Single-cell Perturbation Data](https://mlsanity.com/paper/2210.17283/) (2022)
- [In Search of Insights, Not Magic Bullets: Towards Demystification of the Model Selection Dilemma in Heterogeneous Treatment Effect Estimation](https://mlsanity.com/paper/2302.02923/) (2023)
- [Counterfactual Generative Models for Time-Varying Treatments](https://mlsanity.com/paper/2305.15742/) (2023)
- [Copula-Based Deep Survival Models for Dependent Censoring](https://mlsanity.com/paper/2306.11912/) (2023)
- [Marginal Causal Flows for Validation and Inference](https://mlsanity.com/paper/2411.01295/) (2024)
- [Learning Complementary Policies for Human-AI Teams](https://mlsanity.com/paper/2302.02944/) (2023)
- [RCT Rejection Sampling for Causal Estimation Evaluation](https://mlsanity.com/paper/2307.15176/) (2023)
- [Deep Learning With DAGs](https://mlsanity.com/paper/2401.06864/) (2024)
- [Position: Causal Machine Learning Requires Rigorous Synthetic Experiments for Broader Adoption](https://mlsanity.com/paper/2508.08883/) (2025)
- [RetailSynth: Synthetic Data Generation for Retail AI Systems Evaluation](https://mlsanity.com/paper/2312.14095/) (2023)
- [Variable Importance Matching for Causal Inference](https://mlsanity.com/paper/2302.11715/) (2023)
- [Unveiling the Potential of Robustness in Selecting Conditional Average Treatment Effect Estimators](https://mlsanity.com/paper/2402.18392/) (2024)
- [ProCause: Generating Counterfactual Outcomes to Evaluate Prescriptive Process Monitoring Methods](https://mlsanity.com/paper/2509.00797/) (2025)
- [Generative Conditional Distributions by Neural (Entropic) Optimal Transport](https://mlsanity.com/paper/2406.02317/) (2024)
- [Causal Clustering for Conditional Average Treatment Effects Estimation and Subgroup Discovery](https://mlsanity.com/paper/2509.05775/) (2025)
- [Estimating Heterogeneous Causal Effect on Networks via Orthogonal Learning](https://mlsanity.com/paper/2509.18484/) (2025)
- [Improving Generative Methods for Causal Evaluation via Simulation-Based Inference](https://mlsanity.com/paper/2509.02892/) (2025)
- [A tutorial on discovering and quantifying the effect of latent causal sources of multimodal EHR data](https://mlsanity.com/paper/2510.16026/) (2025)
- [Controllable Generative Sandbox for Causal Inference](https://mlsanity.com/paper/2603.03587/) (2026)

## Abstract (arXiv)

The fundamental challenge of drawing causal inference is that counterfactual outcomes are not fully observed for any unit. Furthermore, in observational studies, treatment assignment is likely to be confounded. Many statistical methods have emerged for causal inference under unconfoundedness conditions given pre-treatment covariates, including propensity score-based methods, prognostic score-based methods, and doubly robust methods. Unfortunately for applied researchers, there is no `one-size-fits-all' causal method that can perform optimally universally. In practice, causal methods are primarily evaluated quantitatively on handcrafted simulated data. Such data-generative procedures can be of limited value because they are typically stylized models of reality. They are simplified for tractability and lack the complexities of real-world data. For applied researchers, it is critical to understand how well a method performs for the data at hand. Our work introduces a deep generative model-based framework, Credence, to validate causal inference methods. The framework's novelty stems from its ability to generate synthetic data anchored at the empirical distribution for the observed sample, and therefore virtually indistinguishable from the latter. The approach allows the user to specify ground truth for the form and magnitude of causal effects and confounding bias as functions of covariates. Thus simulated data sets are used to evaluate the potential performance of various causal estimation methods when applied to data similar to the observed sample. We demonstrate Credence's ability to accurately assess the relative performance of causal estimation techniques in an extensive simulation study and two real-world data applications from Lalonde and Project STAR studies.