# SQLForge: Synthesizing Reliable and Diverse Data to Enhance Text-to-SQL Reasoning in LLMs

arXiv: https://arxiv.org/abs/2505.13725 · HTML: https://mlsanity.com/paper/2505.13725/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning.
- Builds on 16 of its 36 references that are in the map.
- First posted on arXiv 19 May 2025.

## Placed in

- LLM reasoning / [Knowledge- and tool-augmented reasoning](https://mlsanity.com/lr/knowledge-and-tool-augmented-reasoning/)

## Builds on (most cited first, 16 of 16)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [Qwen2 Technical Report](https://mlsanity.com/paper/2407.10671/) (2024)
- [DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence](https://mlsanity.com/paper/2401.14196/) (2024)
- [PaLM 2 Technical Report](https://mlsanity.com/paper/2305.10403/) (2023)
- [Gemma: Open Models Based on Gemini Research and Technology](https://mlsanity.com/paper/2403.08295/) (2024)
- [StarCoder 2 and The Stack v2: The Next Generation](https://mlsanity.com/paper/2402.19173/) (2024)
- [PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Models](https://mlsanity.com/paper/2109.05093/) (2021)
- [Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluation](https://mlsanity.com/paper/2308.15363/) (2023)
- [CHASE-SQL: Multi-Path Reasoning and Preference Optimized Candidate Selection in Text-to-SQL](https://mlsanity.com/paper/2410.01943/) (2024)
- [A Survey on Employing Large Language Models for Text-to-SQL Tasks](https://mlsanity.com/paper/2407.15186/) (2024)
- [A Survey of Text-to-SQL in the Era of LLMs: Where are we, and where are we going?](https://mlsanity.com/paper/2408.05109/) (2024)
- [You Only Read Once (YORO): Learning to Internalize Database Knowledge for Text-to-SQL](https://mlsanity.com/paper/2409.12172/) (2024)
- [AnnotatedTables: A Large Tabular Dataset with Language Model Annotations](https://mlsanity.com/paper/2406.16349/) (2024)

## Built on by (most cited first, 11 of 11)

- [An Efficient and Effective Evaluator for Text2SQL Models on Unseen and Unlabeled Data](https://mlsanity.com/paper/2603.07841/) (2026)
- [CLARITY: A Framework and Benchmark for Conversational Language Ambiguity and Unanswerability in Interactive NL2SQL Systems](https://mlsanity.com/paper/2604.22313/) (2026)
- [All for 1-Bit: Towards Genuine 1-Bit Post-Training Quantization for LLMs](https://mlsanity.com/paper/2609.06161/) (2026)
- [SQL-Exchange: Transforming SQL Queries Across Domains](https://mlsanity.com/paper/2508.07087/) (2025)
- [Beyond SELECT: A Comprehensive Taxonomy-Guided Benchmark for Real-World Text-to-SQL Translation](https://mlsanity.com/paper/2511.13590/) (2025)
- [Learning to Evaluate: Cost-Effective Model Evaluation on Unlabeled Data with Meta-Learning](https://mlsanity.com/paper/2605.23595/) (2026)
- [RingSQL: Schema-Independent Synthetic Data Generation for Text-to-SQL Reinforcement Learning](https://mlsanity.com/paper/2601.05451/) (2026)
- [SemanticAgent: A Semantics-Aware Framework for Text-to-SQL Data Synthesis](https://mlsanity.com/paper/2604.21414/) (2026)
- [Residual Skill Optimization for Text-to-SQL Ensembles](https://mlsanity.com/paper/2605.21792/) (2026)
- [Beyond Static Rules: Automated Discovery of Latent Vulnerabilities in Text-to-SQL](https://mlsanity.com/paper/2607.03833/) (2026)
- [FALCON: A Model and Dataset Agnostic Framework for Synthetic Data Generation for NL2SQL Pairs](https://mlsanity.com/paper/2610.03625/) (2026)

## Abstract (arXiv)

Large Language models (LLMs) have demonstrated significant potential in text-to-SQL reasoning tasks, yet a substantial performance gap persists between existing open-source models and their closed-source counterparts. In this paper, we introduce SQLForge, a novel approach for synthesizing reliable and diverse data to enhance text-to-SQL reasoning in LLMs. We improve data reliability through SQL syntax constraints and SQL-to-question reverse translation, ensuring data logic at both structural and semantic levels. We also propose an SQL template enrichment and iterative data domain exploration mechanism to boost data diversity. Building on the augmented data, we fine-tune a variety of open-source models with different architectures and parameter sizes, resulting in a family of models termed SQLForge-LM. SQLForge-LM achieves the state-of-the-art performance on the widely recognized Spider and BIRD benchmarks among the open-source models. Specifically, SQLForge-LM achieves EX accuracy of 85.7% on Spider Dev and 59.8% on BIRD Dev, significantly narrowing the performance gap with closed-source methods.