TabQueryBench: A Query-Centric Benchmark for Synthetic Tabular Data
Not on the map. Why?
Abstract (arXiv)
Synthetic tabular data support use cases like data sharing, model development under access restrictions, and rapid prototyping of analytical workflows. Modern generative models are evaluated by their statistical similarity, correlation structure, privacy, and downstream machine-learning utility. However, such evaluations leave a gap: they rarely test the structure that matters for analytical queries. We present TabQueryBench, a query-centric benchmark for evaluating synthetic tabular data with SQL-shaped analytical queries. TabQueryBench distills 12 public query sources into 44 reusable query templates and grounds them to each dataset through a policy-guided template-to-SQL pipeline. Across 49 datasets and 11 generative models, it produces more than 100 executable queries per dataset while preserving cross-model comparability. Our evaluation reveals a substantial gap between conventional and query-centric fidelity. Even the strongest model, RealTabFormer, reaches only 0.76 +/- 0.23 query-centric fidelity compared with 1.00 for real data. Failures are most pronounced for high-cardinality attributes and local or tail queries, while model quality also trades off against generation cost. These results show that synthetic data can appear statistically realistic yet still fail to preserve the analytical behavior of real data, motivating query-centric fidelity as a necessary complement to existing evaluation metrics. We open source TabQueryBench to facilitate research in this direction.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 23 of its 69 references that are in the map. First posted on arXiv 4 Jul 2026.
Builds on
23 of 69 references.
References · 23
- 2019Modeling Tabular data using Conditional GANcited by 699
- 2022TabDDPM: Modelling Tabular Data with Diffusion Modelscited by 377
- 2021DatasetGAN: Efficient Labeled Data Factory with Minimal Human Effortcited by 264
- 2023Mixed-Type Tabular Data Synthesis with Score-based Diffusion in Latent Spacecited by 186
- 2017OpenML Benchmarking Suitescited by 178
- 2025TabArena: A Living Benchmark for Machine Learning on Tabular Datacited by 176
- 2023REaLTabFormer: Generating Realistic Relational and Tabular Data using Transformerscited by 94
- 2022DataPerf: Benchmarks for Data-Centric AI Developmentcited by 90
- 2024TabDiff: a Mixed-type Diffusion Model for Tabular Data Generationcited by 63
- 2022Adversarial random forests for density estimation and generative modelingcited by 44
- 2023Synthetic data, real errors: how (not) to publish and use synthetic datacited by 29
- 2023NetDiffusion: Network Data Augmentation Through Protocol-Constrained Traffic Generationcited by 27
11 more
- 2024SynthEval: A Framework for Detailed Utility and Privacy Evaluation of Tabular Synthetic Datacited by 27
- 2023Reimagining Synthetic Tabular Data Generation through Data-Centric AI: A Comprehensive Benchmarkcited by 23
- 2024Systematic Assessment of Tabular Data Synthesiscited by 18
- 2025Exponential Family Variational Flow Matching for Tabular Data Generationcited by 15
- 2024Synthetic Tabular Data Validation: A Divergence-Based Approachcited by 7
- 2024Benchmarking the Fidelity and Utility of Synthetic Relational Datacited by 7
- 2023Diverse Community Data for Benchmarking Data Privacy Algorithmscited by 6
- 2025Benchmarking Synthetic Tabular Data: A Multi-Dimensional Evaluation Frameworkcited by 6
- 2024Feasibility of State Space Models for Network Traffic Generationcited by 4
- 2025Quantifying the Privacy Implications of High-Fidelity Synthetic Network Trafficcited by 3
- 2025High-Fidelity And Complex Test Data Generation For Google SQL Code Generation Servicescited by 1
Built on by
Citing papers per year: 2026: 2.
2 papers.