# Utility Theory of Synthetic Data Generation

arXiv: https://arxiv.org/abs/2305.10015 · HTML: https://mlsanity.com/paper/2305.10015/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 101 references that are in the map.
- First posted on arXiv 17 May 2023.

## Builds on (most cited first, 30 of 30)

- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1611.04076/) (2016)
- [Modeling Tabular data using Conditional GAN](https://mlsanity.com/paper/1907.00503/) (2019)
- [Maximum Likelihood Training of Score-Based Diffusion Models](https://mlsanity.com/paper/2101.09258/) (2021)
- [Nonparametric regression using deep neural networks with ReLU activation function](https://mlsanity.com/paper/1708.06633/) (2017)
- [Unlabeled Data Improves Adversarial Robustness](https://mlsanity.com/paper/1905.13736/) (2019)
- [Synthetic Data from Diffusion Models Improves ImageNet Classification](https://mlsanity.com/paper/2304.08466/) (2023)
- [Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomization](https://mlsanity.com/paper/1804.06516/) (2018)
- [Differentially Private Generative Adversarial Network](https://mlsanity.com/paper/1802.06739/) (2018)
- [Improving Robustness using Generated Data](https://mlsanity.com/paper/2110.09468/) (2021)
- [Synthetic Data Augmentation using GAN for Improved Liver Lesion Classification](https://mlsanity.com/paper/1801.02385/) (2018)
- [How Well Generative Adversarial Networks Learn Distributions](https://mlsanity.com/paper/1811.03179/) (2018)
- [Synthetic Data -- what, why and how?](https://mlsanity.com/paper/2205.03257/) (2022)
- [Optimally tackling covariate shift in RKHS-based nonparametric regression](https://mlsanity.com/paper/2205.02986/) (2022)
- [Optimal Approximation Rates for Deep ReLU Neural Networks on Sobolev and Besov Spaces](https://mlsanity.com/paper/2211.14400/) (2022)
- [Non-Negative Bregman Divergence Minimization for Deep Direct Density Ratio Estimation](https://mlsanity.com/paper/2006.06979/) (2020)
- [Synthetic data, real errors: how (not) to publish and use synthetic data](https://mlsanity.com/paper/2305.09235/) (2023)
- [Downstream Datasets Make Surprisingly Good Pretraining Corpora](https://mlsanity.com/paper/2209.14389/) (2022)
- [Privacy-Preserving Domain Adaptation of Semantic Parsers](https://mlsanity.com/paper/2212.10520/) (2022)
- [Measuring the quality of Synthetic data for use in competitions](https://mlsanity.com/paper/1806.11345/) (2018)
- [Sampling Strategies for GAN Synthetic Data](https://mlsanity.com/paper/1909.04689/) (2019)
- [Improving Adversarial Robustness by Contrastive Guided Diffusion Process](https://mlsanity.com/paper/2210.09643/) (2022)
- [Synthetic Data for Model Selection](https://mlsanity.com/paper/2105.00717/) (2021)
- [Synthetic Data Generation for Fraud Detection using GANs](https://mlsanity.com/paper/2109.12546/) (2021)
- [Statistical Theory of Differentially Private Marginal-based Data Synthesis Algorithms](https://mlsanity.com/paper/2301.08844/) (2023)
- [Relaxed Marginal Consistency for Differentially Private Query Answering](https://mlsanity.com/paper/2109.06153/) (2021)
- [Distribution-Invariant Differential Privacy](https://mlsanity.com/paper/2111.05791/) (2021)
- [SynC: A Unified Framework for Generating Synthetic Population with Gaussian Copula](https://mlsanity.com/paper/1904.07998/) (2019)
- [Analysis of KNN Information Estimators for Smooth Distributions](https://mlsanity.com/paper/1810.11571/) (2018)

## Built on by (most cited first, 8 of 8)

- [A Survey on Statistical Theory of Deep Learning: Approximation, Training Dynamics, and Generative Models](https://mlsanity.com/paper/2401.07187/) (2024)
- [A Theoretical Perspective: How to Prevent Model Collapse in Self-consuming Training Loops](https://mlsanity.com/paper/2502.18865/) (2025)
- [Recursive Learning Without Collapse: A Weighting-Based Stabilization Framework](https://mlsanity.com/paper/2502.18049/) (2025)
- [A Probabilistic Perspective on Model Collapse](https://mlsanity.com/paper/2505.13947/) (2025)
- [GReaTER: Generate Realistic Tabular data after data Enhancement and Reduction](https://mlsanity.com/paper/2503.15564/) (2025)
- [ReTabSyn: Realistic Tabular Data Synthesis via Reinforcement Learning](https://mlsanity.com/paper/2603.10823/) (2026)
- [Enhancing Obsolescence Forecasting with Deep Generative Data Augmentation: A Semi-Supervised Framework for Low-Data Industrial Applications](https://mlsanity.com/paper/2505.01261/) (2025)
- [Task-Conditioned Synthetic Data Generation for Improving Machine Learning Performance in Agricultural Prediction Tasks](https://mlsanity.com/paper/2607.09751/) (2026)

## Abstract (arXiv)

Synthetic data algorithms are widely employed in industries to generate artificial data for downstream learning tasks. While existing research primarily focuses on empirically evaluating utility of synthetic data, its theoretical understanding is largely lacking. This paper bridges the practice-theory gap by establishing relevant utility theory in a statistical learning framework. It considers two utility metrics: generalization and ranking of models trained on synthetic data. The former is defined as the generalization difference between models trained on synthetic and on real data. By deriving analytical bounds for this utility metric, we demonstrate that the synthetic feature distribution does not need to be similar as that of real data for ensuring comparable generalization of synthetic models, provided proper model specifications in downstream learning tasks. The latter utility metric studies the relative performance of models trained on synthetic data. In particular, we discover that the distribution of synthetic data is not necessarily similar as the real one to ensure consistent model comparison. Interestingly, consistent model comparison is still achievable even when synthetic responses are not well generated, as long as downstream models are separable by a generalization gap. Finally, extensive experiments on non-parametric models and deep neural networks have been conducted to validate these theoretical findings.