# SQLBarber: A System Leveraging Large Language Models to Generate Customized and Realistic SQL Workloads

arXiv: https://arxiv.org/abs/2507.06192 · HTML: https://mlsanity.com/paper/2507.06192/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 43 references that are in the map.
- First posted on arXiv 8 Jul 2025.

## Builds on (most cited first, 15 of 15)

- [A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions](https://mlsanity.com/paper/2311.05232/) (2023)
- [LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding](https://mlsanity.com/paper/2308.14508/) (2023)
- [Hallucination is Inevitable: An Innate Limitation of Large Language Models](https://mlsanity.com/paper/2401.11817/) (2024)
- [SMAC3: A Versatile Bayesian Optimization Package for Hyperparameter Optimization](https://mlsanity.com/paper/2109.09831/) (2021)
- [Statistical Aspects of Wasserstein Distances](https://mlsanity.com/paper/1806.05500/) (2018)
- [CodeS: Towards Building Open-source Language Models for Text-to-SQL](https://mlsanity.com/paper/2402.16347/) (2024)
- [GPTuner: A Manual-Reading Database Tuning System via GPT-Guided Bayesian Optimization](https://mlsanity.com/paper/2311.03157/) (2023)
- [DB-BERT: a Database Tuning Tool that "Reads the Manual"](https://mlsanity.com/paper/2112.10925/) (2021)
- [LLM-R2: A Large Language Model Enhanced Rule-based Rewrite System for Boosting Query Efficiency](https://mlsanity.com/paper/2404.12872/) (2024)
- [E2ETune: End-to-End Knob Tuning via Fine-tuned Generative Language Model](https://mlsanity.com/paper/2404.11581/) (2024)
- [QUITE: A Query Rewrite System Beyond Rules with LLM Agents](https://mlsanity.com/paper/2506.07675/) (2025)
- [Is Large Language Model Good at Database Knob Tuning? A Comprehensive Experimental Evaluation](https://mlsanity.com/paper/2408.02213/) (2024)
- [Real-time Workload Pattern Analysis for Large-scale Cloud Databases](https://mlsanity.com/paper/2307.02626/) (2023)
- [Blueprinting the Cloud: Unifying and Automatically Optimizing Cloud Data Infrastructures with BRAD -- Extended Version](https://mlsanity.com/paper/2407.15363/) (2024)
- [Testing Database Engines via Query Plan Guidance](https://mlsanity.com/paper/2312.17510/) (2023)

## Built on by (most cited first, 4 of 4)

- [GenDB: The Next Generation of Query Processing -- Synthesized, Not Engineered](https://mlsanity.com/paper/2603.02081/) (2026)
- [Business Logic-Driven Text-to-SQL Data Synthesis for Business Intelligence](https://mlsanity.com/paper/2601.14518/) (2026)
- [Test-Time Optimization of Physical Query Plans with LLMs](https://mlsanity.com/paper/2602.10387/) (2026)
- [WAter: A Workload-Adaptive Knob Tuning System based on Workload Compression](https://mlsanity.com/paper/2603.28809/) (2026)

## Abstract (arXiv)

Database research and development often require a large number of SQL queries for benchmarking purposes. However, acquiring real-world SQL queries is challenging due to privacy concerns, and existing SQL generation methods are limited in customization and in satisfying realistic constraints. To address this issue, we present SQLBarber, a system based on Large Language Models (LLMs) to generate customized and realistic SQL workloads. SQLBarber (i) eliminates the need for users to manually craft SQL templates in advance, while providing the flexibility to accept natural language specifications to constrain SQL templates, (ii) scales efficiently to generate large volumes of queries matching any user-defined cost distribution (e.g., cardinality and execution plan cost), and (iii) uses execution statistics from Amazon Redshift and Snowflake to derive SQL template specifications and query cost distributions that reflect real-world query characteristics. SQLBarber introduces (i) a declarative interface for users to effortlessly generate customized SQL templates, (ii) an LLM-powered pipeline augmented with a self-correction module that profiles, refines, and prunes SQL templates based on query costs, and (iii) a Bayesian Optimizer to efficiently explore different predicate values and identify a set of queries that satisfy the target cost distribution. We construct and open-source ten benchmarks of varying difficulty levels and target query cost distributions based on real-world statistics from Snowflake and Amazon Redshift. Extensive experiments on these benchmarks show that SQLBarber is the only system that can generate customized SQL templates. It reduces query generation time by one to three orders of magnitude, and significantly improves alignment with the target cost distribution, compared with existing methods.