# Systematic Assessment of Tabular Data Synthesis

arXiv: https://arxiv.org/abs/2402.06806 · HTML: https://mlsanity.com/paper/2402.06806/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Placed in Privacy-preserving tabular data synthesis, one of 13 directions in Synthetic data and self-improvement for LLMs.
- Builds on 42 of its 92 references that are in the map.
- First posted on arXiv 9 Feb 2024.

## Placed in

- Synthetic data and self-improvement for LLMs / [Privacy-preserving tabular data synthesis](https://mlsanity.com/synth/privacy-preserving-tabular-data-synthesis/)

## Builds on (most cited first, 30 of 42)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Membership Inference Attacks From First Principles](https://mlsanity.com/paper/2112.03570/) (2021)
- [Computational Optimal Transport](https://mlsanity.com/paper/1803.00567/) (2018)
- [Revisiting Deep Learning Models for Tabular Data](https://mlsanity.com/paper/2106.11959/) (2021)
- [Extracting Training Data from Diffusion Models](https://mlsanity.com/paper/2301.13188/) (2023)
- [Modeling Tabular data using Conditional GAN](https://mlsanity.com/paper/1907.00503/) (2019)
- [CVXPY: A Python-Embedded Modeling Language for Convex Optimization](https://mlsanity.com/paper/1603.00943/) (2016)
- [Does Learning Require Memorization? A Short Tale about a Long Tail](https://mlsanity.com/paper/1906.05271/) (2019)
- [TabDDPM: Modelling Tabular Data with Diffusion Models](https://mlsanity.com/paper/2209.15421/) (2022)
- [Language Models are Realistic Tabular Data Generators](https://mlsanity.com/paper/2210.06280/) (2022)
- [A note on the evaluation of generative models](https://mlsanity.com/paper/1511.01844/) (2015)
- [Scalable Private Learning with PATE](https://mlsanity.com/paper/1802.08908/) (2018)
- [How Faithful is your Synthetic Data? Sample-level Metrics for Evaluating and Auditing Generative Models](https://mlsanity.com/paper/2102.08921/) (2021)
- [Mixed-Type Tabular Data Synthesis with Score-based Diffusion in Latent Space](https://mlsanity.com/paper/2310.09656/) (2023)
- [Data Synthesis based on Generative Adversarial Networks](https://mlsanity.com/paper/1806.03384/) (2018)
- [CTAB-GAN: Effective Table Data Synthesizing](https://mlsanity.com/paper/2102.08369/) (2021)
- [Are Diffusion Models Vulnerable to Membership Inference Attacks?](https://mlsanity.com/paper/2302.01316/) (2023)
- [Synthetic Data -- Anonymisation Groundhog Day](https://mlsanity.com/paper/2011.07018/) (2020)
- [Winning the NIST Contest: A scalable and general approach to differentially private synthetic data](https://mlsanity.com/paper/2108.04978/) (2021)
- [REaLTabFormer: Generating Realistic Relational and Tabular Data using Transformers](https://mlsanity.com/paper/2302.02041/) (2023)
- [AIM: An Adaptive and Iterative Mechanism for Differentially Private Synthetic Data](https://mlsanity.com/paper/2201.12677/) (2022)
- [STaSy: Score-based Tabular data Synthesis](https://mlsanity.com/paper/2210.04018/) (2022)
- [CoDi: Co-evolving Contrastive Diffusion Models for Mixed-type Tabular Synthesis](https://mlsanity.com/paper/2304.12654/) (2023)
- [Synthcity: facilitating innovative use cases of synthetic data in different data modalities](https://mlsanity.com/paper/2301.07573/) (2023)
- [CTAB-GAN+: Enhancing Tabular Data Synthesis](https://mlsanity.com/paper/2204.00401/) (2022)
- [Synthetic Data -- what, why and how?](https://mlsanity.com/paper/2205.03257/) (2022)
- [Membership Inference Attacks against Synthetic Data through Overfitting Detection](https://mlsanity.com/paper/2302.12580/) (2023)

## Built on by (most cited first, 18 of 18)

- [Privacy Vulnerabilities in Marginals-based Synthetic Data](https://mlsanity.com/paper/2410.05506/) (2024)
- [Imitative Membership Inference Attack](https://mlsanity.com/paper/2509.06796/) (2025)
- [Membership Inference Attacks on Tokenizers of Large Language Models](https://mlsanity.com/paper/2510.05699/) (2025)
- [Differentially Private Synthetic Data via APIs 4: Tabular Data](https://mlsanity.com/paper/2606.08259/) (2026)
- [TabQueryBench: A Query-Centric Benchmark for Synthetic Tabular Data](https://mlsanity.com/paper/2607.03926/) (2026)
- [Six Levels of Privacy: A Framework for Financial Synthetic Data](https://mlsanity.com/paper/2403.14724/) (2024)
- [PrivORL: Differentially Private Synthetic Dataset for Offline Reinforcement Learning](https://mlsanity.com/paper/2512.07342/) (2025)
- [Rethinking Anonymity Claims in Synthetic Data Generation: A Model-Centric Privacy Attack Perspective](https://mlsanity.com/paper/2601.22434/) (2026)
- [FHAIM: Fully Homomorphic AIM For Private Synthetic Data Generation](https://mlsanity.com/paper/2602.05838/) (2026)
- [Generative Augmentation of Imbalanced Flight Records for Flight Diversion Prediction: A Multi-objective Optimisation Framework](https://mlsanity.com/paper/2604.20288/) (2026)
- [Generating Reliable Synthetic Clinical Trial Data: The Role of Hyperparameter Optimization and Domain Constraints](https://mlsanity.com/paper/2505.05019/) (2025)
- [Synthetic Homes: A Multimodal Generative AI Pipeline for Residential Building Data Generation under Data Scarcity](https://mlsanity.com/paper/2509.09794/) (2025)
- [SYNTHONY: A Stress-Aware, Intent-Conditioned Agent for Deep Tabular Generative Models Selection](https://mlsanity.com/paper/2604.00293/) (2026)
- [Evaluating quality in synthetic data generation for large tabular health datasets](https://mlsanity.com/paper/2604.15961/) (2026)
- [Seq2Synth: Benchmarking Temporal Fidelity in Synthetic Sequential Tabular Data](https://mlsanity.com/paper/2607.15606/) (2026)
- [PEARL: A Task-Aware Framework for Evaluating Differentially Private Synthetic Educational Data](https://mlsanity.com/paper/2609.10612/) (2026)
- [MIND: Marginal-Invariant Neural Dependency Diffusion for Mixed-Type Tabular Generation](https://mlsanity.com/paper/2609.39628/) (2026)
- [Below what training size do deep tabular generators stop beating trivial baselines? A preregistered benchmark on a size ladder of clinical and standard datasets](https://mlsanity.com/paper/2610.03500/) (2026)

## Abstract (arXiv)

Data synthesis has been advocated as an important approach for utilizing data while protecting data privacy. In recent years, a plethora of tabular data synthesis algorithms (i.e., synthesizers) have been proposed. Some synthesizers satisfy Differential Privacy, while others aim to provide privacy in a heuristic fashion. A comprehensive understanding of the strengths and weaknesses of these synthesizers remains elusive due to drawbacks in evaluation metrics and missing head-to-head comparisons of newly developed synthesizers that take advantage of diffusion models and large language models with state-of-the-art statistical synthesizers. In this paper, we present a systematic evaluation framework for assessing tabular data synthesis algorithms. Specifically, we examine and critique existing evaluation metrics, and introduce a set of new metrics in terms of fidelity, privacy, and utility to address their limitations. We conducted extensive evaluations of 8 different types of synthesizers on 12 real-world datasets and identified some interesting findings, which offer new directions for privacy-preserving data synthesis.