# Benchmarking Synthetic Tabular Data: A Multi-Dimensional Evaluation Framework

arXiv: https://arxiv.org/abs/2504.01908 · HTML: https://mlsanity.com/paper/2504.01908/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 14 references that are in the map.
- First posted on arXiv 2 Apr 2025.

## Builds on (most cited first, 4 of 4)

- [How Faithful is your Synthetic Data? Sample-level Metrics for Evaluating and Auditing Generative Models](https://mlsanity.com/paper/2102.08921/) (2021)
- [TabSynDex: A Universal Metric for Robust Evaluation of Synthetic Tabular Data](https://mlsanity.com/paper/2207.05295/) (2022)
- [Synthetic Data for Social Good](https://mlsanity.com/paper/1710.08874/) (2017)
- [Benchmarking the Fidelity and Utility of Synthetic Relational Data](https://mlsanity.com/paper/2410.03411/) (2024)

## Built on by (most cited first, 6 of 6)

- [TabQueryBench: A Query-Centric Benchmark for Synthetic Tabular Data](https://mlsanity.com/paper/2607.03926/) (2026)
- [Democratizing Tabular Data Access with an Openunicodex2013Source Syntheticunicodex2013Data SDK](https://mlsanity.com/paper/2508.00718/) (2025)
- [SHAP Distance: An Explainability-Aware Metric for Evaluating the Semantic Fidelity of Synthetic Tabular Data](https://mlsanity.com/paper/2511.17590/) (2025)
- [No Free Lunch for Synthetic Images under Data Scarcity Conditions](https://mlsanity.com/paper/2606.07640/) (2026)
- [Declarative Outcome-Conformant Synthesis: Exact, Closed-Form Specification Satisfaction and a Conformance Benchmark](https://mlsanity.com/paper/2606.08736/) (2026)
- [Position: Privacy Is a Claim, Not a Property of Synthetic Data](https://mlsanity.com/paper/2609.01273/) (2026)

## Abstract (arXiv)

Evaluating the quality of synthetic data remains a key challenge for ensuring privacy and utility in data-driven research. In this work, we present an evaluation framework that quantifies how well synthetic data replicates original distributional properties while ensuring privacy. The proposed approach employs a holdout-based benchmarking strategy that facilitates quantitative assessment through low- and high-dimensional distribution comparisons, embedding-based similarity measures, and nearest-neighbor distance metrics. The framework supports various data types and structures, including sequential and contextual information, and enables interpretable quality diagnostics through a set of standardized metrics. These contributions aim to support reproducibility and methodological consistency in benchmarking of synthetic data generation techniques. The code of the framework is available at https://github.com/mostly-ai/mostlyai-qa.