# Generative Synthetic Data for Causal Inference: Pitfalls, Remedies, and Opportunities

arXiv: https://arxiv.org/abs/2604.23904 · HTML: https://mlsanity.com/paper/2604.23904/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Privacy-preserving tabular data synthesis, one of 13 directions in Synthetic data and self-improvement for LLMs.
- Builds on 17 of its 32 references that are in the map.
- First posted on arXiv 26 Apr 2026.

## Placed in

- Synthetic data and self-improvement for LLMs / [Privacy-preserving tabular data synthesis](https://mlsanity.com/synth/privacy-preserving-tabular-data-synthesis/)

## Builds on (most cited first, 17 of 17)

- [Modeling Tabular data using Conditional GAN](https://mlsanity.com/paper/1907.00503/) (2019)
- [Quasi-Oracle Estimation of Heterogeneous Treatment Effects](https://mlsanity.com/paper/1712.04912/) (2017)
- [Language Models are Realistic Tabular Data Generators](https://mlsanity.com/paper/2210.06280/) (2022)
- [On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Survey](https://mlsanity.com/paper/2406.15126/) (2024)
- [Automated versus do-it-yourself methods for causal inference: Lessons learned from a data analysis competition](https://mlsanity.com/paper/1707.02641/) (2017)
- [Synthetic Data Generation with Large Language Models for Text Classification: Potential and Limitations](https://mlsanity.com/paper/2310.07849/) (2023)
- [Balancing Covariates via Propensity Score Weighting](https://mlsanity.com/paper/1609.07494/) (2016)
- [Large Language Models and Causal Inference in Collaboration: A Survey](https://mlsanity.com/paper/2403.09606/) (2024)
- [Synthetic Oversampling: Theory and A Practical Approach Using LLMs to Address Data Imbalance](https://mlsanity.com/paper/2406.03628/) (2024)
- [Efficient Randomized Experiments Using Foundation Models](https://mlsanity.com/paper/2502.04262/) (2025)
- [Propensity score weighting under limited overlap and model misspecification](https://mlsanity.com/paper/2006.04038/) (2020)
- [Bias-Corrected Data Synthesis for Imbalanced Learning](https://mlsanity.com/paper/2510.26046/) (2025)
- [Improving the Generation and Evaluation of Synthetic Data for Downstream Medical Causal Inference](https://mlsanity.com/paper/2510.18768/) (2025)
- [Addressing Extreme Propensity Scores in Estimating Counterfactual Survival Functions via the Overlap Weights](https://mlsanity.com/paper/2108.04394/) (2021)
- [Generative modeling for the bootstrap](https://mlsanity.com/paper/2602.17052/) (2026)
- [Investigating Targeting Strategies and Truncation in TMLE for the Average Treatment Effect under Practical Positivity Violations](https://mlsanity.com/paper/2604.20059/) (2026)
- [Residual Feature Integration is Sufficient to Prevent Negative Transfer](https://mlsanity.com/paper/2505.11771/) (2025)

## Built on by (most cited first, 2 of 2)

- [When Does Synthetic Data Augmentation Improve Score-Based Imbalanced Classification?](https://mlsanity.com/paper/2606.26053/) (2026)
- [Beyond Distributional Fidelity: Causal-Penalized Diffusion for Synthetic Tabular Data](https://mlsanity.com/paper/2610.11407/) (2026)

## Abstract (arXiv)

Synthetic tabular data are often evaluated by distributional similarity, privacy distance, or train-on-synthetic-test-on-real predictive performance, but these criteria do not ensure validity for causal inference. We show that fully generative tabular synthesizers, including GAN- and LLM-based models, can preserve predictive utility while distorting average treatment effect (ATE) estimates. The failure is structural: ATE preservation requires both a realistic covariate law and an accurate treatment-effect contrast, whereas prediction loss penalizes treatment-effect error only through an overlap-weighted term. Thus, under imbalance or limited overlap, a generator may reproduce dominant observed outcomes while underlearning intervention-relevant contrasts. We formalize this mismatch through sensitivity and loss-decomposition results. Motivated by this causal analysis and intuition, we propose a hybrid synthetic-data framework for causal inference that generates covariates while modeling treatment and outcome mechanisms separately. We evaluate the framework in three settings: ATE preservation under fully generative versus hybrid synthesis, augmentation for practical positivity problems, and diagnostic simulation engines for comparing OR, IPW, AIPW, and TMLE before real-data analysis. We also stress-test the hybrid construction across settings that vary overlap, covariate dimension, seed sample size, and treatment-effect complexity, including a logistic outcome-model misspecification check. Across controlled simulation experiments, hybrid synthesis improves causal fidelity relative to fully generative baselines; the ACTG application shows improved predictive fidelity and potential for finite-sample estimator benchmarking. LLM-based hybrid synthesis is often more faithful than CTGAN in settings where causal fidelity can be assessed.