mlsanityData to 7 Oct 2026

Systematic Assessment of Tabular Data Synthesis

arXiv 2402.06806 ↗First posted 9 Feb 2024cs.CR cs.DB cs.LGcited by 17

On the map: Synthetic data and self-improvement for LLMs → Privacy-preserving tabular data synthesis

Abstract (arXiv)

Data synthesis has been advocated as an important approach for utilizing data while protecting data privacy. In recent years, a plethora of tabular data synthesis algorithms (i.e., synthesizers) have been proposed. Some synthesizers satisfy Differential Privacy, while others aim to provide privacy in a heuristic fashion. A comprehensive understanding of the strengths and weaknesses of these synthesizers remains elusive due to drawbacks in evaluation metrics and missing head-to-head comparisons of newly developed synthesizers that take advantage of diffusion models and large language models with state-of-the-art statistical synthesizers. In this paper, we present a systematic evaluation framework for assessing tabular data synthesis algorithms. Specifically, we examine and critique existing evaluation metrics, and introduce a set of new metrics in terms of fidelity, privacy, and utility to address their limitations. We conducted extensive evaluations of 8 different types of synthesizers on 12 real-world datasets and identified some interesting findings, which offer new directions for privacy-preserving data synthesis.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Placed in Privacy-preserving tabular data synthesis, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 42 of its 92 references that are in the map. First posted on arXiv 9 Feb 2024.

Builds on

42 of 92 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Mixed-Type Tabular Data Synthesis with Score-based Diffusion in Latent Space14 Oct

Other references · 41

  1. 2017Attention Is All You Needcited by 59,932
  2. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
  3. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,455
  4. 2017Improved Training of Wasserstein GANscited by 2,223
  5. 2021Membership Inference Attacks From First Principlescited by 839
  6. 2018Computational Optimal Transportcited by 765
  7. 2021Revisiting Deep Learning Models for Tabular Datacited by 725
  8. 2023Extracting Training Data from Diffusion Modelscited by 723
  9. 2019Modeling Tabular data using Conditional GANcited by 695
  10. 2016CVXPY: A Python-Embedded Modeling Language for Convex Optimizationcited by 633
  11. 2019Does Learning Require Memorization? A Short Tale about a Long Tailcited by 452
  12. 2022TabDDPM: Modelling Tabular Data with Diffusion Modelscited by 373
29 more
  1. 2015A note on the evaluation of generative modelscited by 291
  2. 2022Language Models are Realistic Tabular Data Generatorscited by 290
  3. 2018Scalable Private Learning with PATEcited by 266
  4. 2021How Faithful is your Synthetic Data? Sample-level Metrics for Evaluating and Auditing Generative Modelscited by 193
  5. 2018Data Synthesis based on Generative Adversarial Networkscited by 184
  6. 2021CTAB-GAN: Effective Table Data Synthesizingcited by 156
  7. 2023Are Diffusion Models Vulnerable to Membership Inference Attacks?cited by 135
  8. 2020Synthetic Data -- Anonymisation Groundhog Daycited by 105
  9. 2021Winning the NIST Contest: A scalable and general approach to differentially private synthetic datacited by 101
  10. 2023REaLTabFormer: Generating Realistic Relational and Tabular Data using Transformerscited by 93
  11. 2022AIM: An Adaptive and Iterative Mechanism for Differentially Private Synthetic Datacited by 88
  12. 2022STaSy: Score-based Tabular data Synthesiscited by 85
  13. 2023CoDi: Co-evolving Contrastive Diffusion Models for Mixed-type Tabular Synthesiscited by 78
  14. 2023Synthcity: facilitating innovative use cases of synthetic data in different data modalitiescited by 76
  15. 2022CTAB-GAN+: Enhancing Tabular Data Synthesiscited by 70
  16. 2022Synthetic Data -- what, why and how?cited by 66
  17. 2023Membership Inference Attacks against Synthetic Data through Overfitting Detectioncited by 66
  18. 2022TAPAS: a Toolbox for Adversarial Privacy Auditing of Synthetic Datacited by 56
  19. 2022Score-based Generative Modeling Secretly Minimizes the Wasserstein Distancecited by 50
  20. 2021Robin Hood and Matthew Effects: Differential Privacy Has Disparate Impact on Synthetic Datacited by 48
  21. 2022Are Attribute Inference Attacks Just Imputation?cited by 48
  22. 2022A Unified Framework for Quantifying Privacy Risk in Synthetic Datacited by 45
  23. 2023Achilles' Heels: Vulnerable Record Identification in Synthetic Data Publishingcited by 28
  24. 2022TabSynDex: A Universal Metric for Robust Evaluation of Synthetic Tabular Datacited by 27
  25. 2023A Linear Reconstruction Approach for Attribute Inference Attacks against Synthetic Datacited by 23
  26. 2021Fidelity and Privacy of Synthetic Medical Datacited by 16
  27. 2023The Inadequacy of Similarity-based Privacy Metrics: Privacy Attacks against "Truly Anonymous" Synthetic Datasetscited by 14
  28. 2021On Utility and Privacy in Synthetic Genomic Datacited by 8
  29. 2021DPSyn: Experiences in the NIST Differential Privacy Data Synthesis Challengescited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 5; 2026: 10.

17 papers.

Sort:

By area:

  1. 2024Privacy Vulnerabilities in Marginals-based Synthetic Datacited by 10
  2. 2025Membership Inference Attacks on Tokenizers of Large Language Modelscited by 3
  3. 2025Imitative Membership Inference Attackcited by 2
  4. 2026Differentially Private Synthetic Data via APIs 4: Tabular Datacited by 2
  5. 2026TabQueryBench: A Query-Centric Benchmark for Synthetic Tabular Datacited by 2
  6. 2024Six Levels of Privacy: A Framework for Financial Synthetic Datacited by 1
  7. 2025PrivORL: Differentially Private Synthetic Dataset for Offline Reinforcement Learningcited by 1
  8. 2026Rethinking Anonymity Claims in Synthetic Data Generation: A Model-Centric Privacy Attack Perspectivecited by 1
  9. 2026FHAIM: Fully Homomorphic AIM For Private Synthetic Data Generationcited by 1
  10. 2026Generative Augmentation of Imbalanced Flight Records for Flight Diversion Prediction: A Multi-objective Optimisation Frameworkcited by 1
  11. 2025Generating Reliable Synthetic Clinical Trial Data: The Role of Hyperparameter Optimization and Domain Constraintscited by 0
  12. 2025Synthetic Homes: A Multimodal Generative AI Pipeline for Residential Building Data Generation under Data Scarcitycited by 0
5 more
  1. 2026SYNTHONY: A Stress-Aware, Intent-Conditioned Agent for Deep Tabular Generative Models Selectioncited by 0
  2. 2026Evaluating quality in synthetic data generation for large tabular health datasetscited by 0
  3. 2026Seq2Synth: Benchmarking Temporal Fidelity in Synthetic Sequential Tabular Datacited by 0
  4. 2026PEARL: A Task-Aware Framework for Evaluating Differentially Private Synthetic Educational Datacited by 0
  5. 2026MIND: Marginal-Invariant Neural Dependency Diffusion for Mixed-Type Tabular Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Creation begins with understanding: LLMs as strategy designers for privacy-preserving tabular data synthesis
  2. 2024Tabular Data Synthesis with Differential Privacy: A Survey
  3. 2024Differentially Private Tabular Data Synthesis using Large Language Models
  4. 2025Flow Matching for Tabular Data Synthesis
  5. 2025Membership Inference over Diffusion-models-based Synthetic Tabular Data
  6. 2024HARMONIC: Harnessing LLMs for Tabular Data Synthesis and Privacy Protection
  7. 2023TabuLa: Harnessing Language Models for Tabular Data Synthesis
  8. 2025Winning the MIDST Challenge: New Membership Inference Attacks on Diffusion Models for Tabular Data Synthesis