# Benchmarking the Fidelity and Utility of Synthetic Relational Data

arXiv: https://arxiv.org/abs/2410.03411 · HTML: https://mlsanity.com/paper/2410.03411/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 38 references that are in the map.
- First posted on arXiv 4 Oct 2024.

## Builds on (most cited first, 12 of 12)

- [Modeling Tabular data using Conditional GAN](https://mlsanity.com/paper/1907.00503/) (2019)
- [Neural Spline Flows](https://mlsanity.com/paper/1906.04032/) (2019)
- [Deep Neural Networks and Tabular Data: A Survey](https://mlsanity.com/paper/2110.01889/) (2021)
- [Revisiting Classifier Two-Sample Tests](https://mlsanity.com/paper/1610.06545/) (2016)
- [REaLTabFormer: Generating Realistic Relational and Tabular Data using Transformers](https://mlsanity.com/paper/2302.02041/) (2023)
- [Synthcity: facilitating innovative use cases of synthetic data in different data modalities](https://mlsanity.com/paper/2301.07573/) (2023)
- [Relational Deep Learning: Graph Representation Learning on Relational Databases](https://mlsanity.com/paper/2312.04615/) (2023)
- [Classification accuracy as a proxy for two sample testing](https://mlsanity.com/paper/1602.02210/) (2016)
- [ClavaDDPM: Multi-relational Data Synthesis with Cluster-guided Diffusion Models](https://mlsanity.com/paper/2405.17724/) (2024)
- [Reimagining Synthetic Tabular Data Generation through Data-Centric AI: A Comprehensive Benchmark](https://mlsanity.com/paper/2310.16981/) (2023)
- [Row Conditional-TGAN for generating synthetic relational databases](https://mlsanity.com/paper/2211.07588/) (2022)
- [Generating Realistic Synthetic Relational Data through Graph Variational Autoencoders](https://mlsanity.com/paper/2211.16889/) (2022)

## Built on by (most cited first, 7 of 7)

- [Benchmarking Synthetic Tabular Data: A Multi-Dimensional Evaluation Framework](https://mlsanity.com/paper/2504.01908/) (2025)
- [RelDiff: Relational Data Generative Modeling with Graph-Based Diffusion Models](https://mlsanity.com/paper/2506.00710/) (2025)
- [Graph-Conditional Flow Matching for Relational Data Generation](https://mlsanity.com/paper/2505.15668/) (2025)
- [The Era by Eon Benchmark: A Generated Enterprise Estate with Exact Ground Truth for Benchmarking LLM Agents](https://mlsanity.com/paper/2609.09853/) (2026)
- [TabQueryBench: A Query-Centric Benchmark for Synthetic Tabular Data](https://mlsanity.com/paper/2607.03926/) (2026)
- [Measuring the Dependency Gap: Diagnosing Inter-Column Fidelity in Tabular Generative Models](https://mlsanity.com/paper/2607.21636/) (2026)
- [Beyond Marginals: A Multi-Dimensional Evaluation Framework for Multi-Table Synthetic Data Generation](https://mlsanity.com/paper/2610.06854/) (2026)

## Abstract (arXiv)

Synthesizing relational data has started to receive more attention from researchers, practitioners, and industry. The task is more difficult than synthesizing a single table due to the added complexity of relationships between tables. For the same reason, benchmarking methods for synthesizing relational data introduces new challenges. Our work is motivated by a lack of an empirical evaluation of state-of-the-art methods and by gaps in the understanding of how such an evaluation should be done. We review related work on relational data synthesis, common benchmarking datasets, and approaches to measuring the fidelity and utility of synthetic data. We combine the best practices and a novel robust detection approach into a benchmarking tool and use it to compare six methods, including two commercial tools. While some methods are better than others, no method is able to synthesize a dataset that is indistinguishable from original data. For utility, we typically observe moderate correlation between real and synthetic data for both model predictive performance and feature importance.