# TabQueryBench: A Query-Centric Benchmark for Synthetic Tabular Data

arXiv: https://arxiv.org/abs/2607.03926 · HTML: https://mlsanity.com/paper/2607.03926/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 69 references that are in the map.
- First posted on arXiv 4 Jul 2026.

## Builds on (most cited first, 23 of 23)

- [Modeling Tabular data using Conditional GAN](https://mlsanity.com/paper/1907.00503/) (2019)
- [TabDDPM: Modelling Tabular Data with Diffusion Models](https://mlsanity.com/paper/2209.15421/) (2022)
- [DatasetGAN: Efficient Labeled Data Factory with Minimal Human Effort](https://mlsanity.com/paper/2104.06490/) (2021)
- [Mixed-Type Tabular Data Synthesis with Score-based Diffusion in Latent Space](https://mlsanity.com/paper/2310.09656/) (2023)
- [OpenML Benchmarking Suites](https://mlsanity.com/paper/1708.03731/) (2017)
- [TabArena: A Living Benchmark for Machine Learning on Tabular Data](https://mlsanity.com/paper/2506.16791/) (2025)
- [REaLTabFormer: Generating Realistic Relational and Tabular Data using Transformers](https://mlsanity.com/paper/2302.02041/) (2023)
- [DataPerf: Benchmarks for Data-Centric AI Development](https://mlsanity.com/paper/2207.10062/) (2022)
- [TabDiff: a Mixed-type Diffusion Model for Tabular Data Generation](https://mlsanity.com/paper/2410.20626/) (2024)
- [Adversarial random forests for density estimation and generative modeling](https://mlsanity.com/paper/2205.09435/) (2022)
- [Synthetic data, real errors: how (not) to publish and use synthetic data](https://mlsanity.com/paper/2305.09235/) (2023)
- [NetDiffusion: Network Data Augmentation Through Protocol-Constrained Traffic Generation](https://mlsanity.com/paper/2310.08543/) (2023)
- [SynthEval: A Framework for Detailed Utility and Privacy Evaluation of Tabular Synthetic Data](https://mlsanity.com/paper/2404.15821/) (2024)
- [Reimagining Synthetic Tabular Data Generation through Data-Centric AI: A Comprehensive Benchmark](https://mlsanity.com/paper/2310.16981/) (2023)
- [Systematic Assessment of Tabular Data Synthesis](https://mlsanity.com/paper/2402.06806/) (2024)
- [Exponential Family Variational Flow Matching for Tabular Data Generation](https://mlsanity.com/paper/2506.05940/) (2025)
- [Synthetic Tabular Data Validation: A Divergence-Based Approach](https://mlsanity.com/paper/2405.07822/) (2024)
- [Benchmarking the Fidelity and Utility of Synthetic Relational Data](https://mlsanity.com/paper/2410.03411/) (2024)
- [Diverse Community Data for Benchmarking Data Privacy Algorithms](https://mlsanity.com/paper/2306.13216/) (2023)
- [Benchmarking Synthetic Tabular Data: A Multi-Dimensional Evaluation Framework](https://mlsanity.com/paper/2504.01908/) (2025)
- [Feasibility of State Space Models for Network Traffic Generation](https://mlsanity.com/paper/2406.02784/) (2024)
- [Quantifying the Privacy Implications of High-Fidelity Synthetic Network Traffic](https://mlsanity.com/paper/2511.20497/) (2025)
- [High-Fidelity And Complex Test Data Generation For Google SQL Code Generation Services](https://mlsanity.com/paper/2504.17203/) (2025)

## Built on by (most cited first, 2 of 2)

- [The Era by Eon Benchmark: A Generated Enterprise Estate with Exact Ground Truth for Benchmarking LLM Agents](https://mlsanity.com/paper/2609.09853/) (2026)
- [Generating a Consistent Enterprise: Synthesis and Reference-Free Evaluation of Multi-System Business Data](https://mlsanity.com/paper/2609.11286/) (2026)

## Abstract (arXiv)

Synthetic tabular data support use cases like data sharing, model development under access restrictions, and rapid prototyping of analytical workflows. Modern generative models are evaluated by their statistical similarity, correlation structure, privacy, and downstream machine-learning utility. However, such evaluations leave a gap: they rarely test the structure that matters for analytical queries. We present TabQueryBench, a query-centric benchmark for evaluating synthetic tabular data with SQL-shaped analytical queries. TabQueryBench distills 12 public query sources into 44 reusable query templates and grounds them to each dataset through a policy-guided template-to-SQL pipeline. Across 49 datasets and 11 generative models, it produces more than 100 executable queries per dataset while preserving cross-model comparability. Our evaluation reveals a substantial gap between conventional and query-centric fidelity. Even the strongest model, RealTabFormer, reaches only 0.76 +/- 0.23 query-centric fidelity compared with 1.00 for real data. Failures are most pronounced for high-cardinality attributes and local or tail queries, while model quality also trades off against generation cost. These results show that synthetic data can appear statistically realistic yet still fail to preserve the analytical behavior of real data, motivating query-centric fidelity as a necessary complement to existing evaluation metrics. We open source TabQueryBench to facilitate research in this direction.