# DTS-SQL: Decomposed Text-to-SQL with Small Large Language Models

arXiv: https://arxiv.org/abs/2402.01117 · HTML: https://mlsanity.com/paper/2402.01117/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 37 references that are in the map.
- First posted on arXiv 2 Feb 2024.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning](https://mlsanity.com/paper/2307.08691/) (2023)
- [Learning to summarize from human feedback](https://mlsanity.com/paper/2009.01325/) (2020)
- [Fine-Tuning Language Models from Human Preferences](https://mlsanity.com/paper/1909.08593/) (2019)
- [Multitask Prompted Training Enables Zero-Shot Task Generalization](https://mlsanity.com/paper/2110.08207/) (2021)
- [LaMDA: Language Models for Dialog Applications](https://mlsanity.com/paper/2201.08239/) (2022)
- [Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task](https://mlsanity.com/paper/1809.08887/) (2018)
- [Cross-Task Generalization via Natural Language Crowdsourcing Instructions](https://mlsanity.com/paper/2104.08773/) (2021)
- [Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs](https://mlsanity.com/paper/2305.03111/) (2023)
- [RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedback](https://mlsanity.com/paper/2309.00267/) (2023)
- [DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correction](https://mlsanity.com/paper/2304.11015/) (2023)
- [RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers](https://mlsanity.com/paper/1911.04942/) (2019)
- [Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluation](https://mlsanity.com/paper/2308.15363/) (2023)
- [Towards Complex Text-to-SQL in Cross-Domain Database with Intermediate Representation](https://mlsanity.com/paper/1905.08205/) (2019)
- [RESDSQL: Decoupling Schema Linking and Skeleton Parsing for Text-to-SQL](https://mlsanity.com/paper/2302.05965/) (2023)
- [C3: Zero-shot Text-to-SQL with ChatGPT](https://mlsanity.com/paper/2307.07306/) (2023)
- [LGESQL: Line Graph Enhanced Text-to-SQL Model with Mixed Local and Non-Local Relations](https://mlsanity.com/paper/2106.01093/) (2021)
- [Towards Robustness of Text-to-SQL Models against Synonym Substitution](https://mlsanity.com/paper/2106.01065/) (2021)
- [Graphix-T5: Mixing Pre-Trained Transformers with Graph-Aware Layers for Text-to-SQL Parsing](https://mlsanity.com/paper/2301.07507/) (2023)
- [RASAT: Integrating Relational Structures into Pretrained Seq2Seq Model for Text-to-SQL](https://mlsanity.com/paper/2205.06983/) (2022)
- [SADGA: Structure-Aware Dual Graph Aggregation Network for Text-to-SQL](https://mlsanity.com/paper/2111.00653/) (2021)
- [SeaD: End-to-end Text-to-SQL Generation with Schema-aware Denoising](https://mlsanity.com/paper/2105.07911/) (2021)
- [Blar-SQL: Faster, Stronger, Smaller NL2SQL](https://mlsanity.com/paper/2401.02997/) (2024)

## Built on by (most cited first, 30 of 45)

- [CHESS: Contextual Harnessing for Efficient SQL Synthesis](https://mlsanity.com/paper/2405.16755/) (2024)
- [CHASE-SQL: Multi-Path Reasoning and Preference Optimized Candidate Selection in Text-to-SQL](https://mlsanity.com/paper/2410.01943/) (2024)
- [Next-Generation Database Interfaces: A Survey of LLM-based Text-to-SQL](https://mlsanity.com/paper/2406.08426/) (2024)
- [A Survey on Employing Large Language Models for Text-to-SQL Tasks](https://mlsanity.com/paper/2407.15186/) (2024)
- [Reasoning-SQL: Reinforcement Learning with SQL Tailored Partial Rewards for Reasoning-Enhanced Text-to-SQL](https://mlsanity.com/paper/2503.23157/) (2025)
- [A Survey of Text-to-SQL in the Era of LLMs: Where are we, and where are we going?](https://mlsanity.com/paper/2408.05109/) (2024)
- [OpenSearch-SQL: Enhancing Text-to-SQL with Dynamic Few-shot and Consistency Alignment](https://mlsanity.com/paper/2502.14913/) (2025)
- [RSL-SQL: Robust Schema Linking in Text-to-SQL Generation](https://mlsanity.com/paper/2411.00073/) (2024)
- [TableRAG: Million-Token Table Understanding with Language Models](https://mlsanity.com/paper/2410.04739/) (2024)
- [XiYan-SQL: A Novel Multi-Generator Framework For Text-to-SQL](https://mlsanity.com/paper/2507.04701/) (2025)
- [MSc-SQL: Multi-Sample Critiquing Small Language Models For Text-To-SQL Translation](https://mlsanity.com/paper/2410.12916/) (2024)
- [SQL-o1: A Self-Reward Heuristic Dynamic Search Method for Text-to-SQL](https://mlsanity.com/paper/2502.11741/) (2025)
- [BIRD-INTERACT: Re-imagining Text-to-SQL Evaluation for Large Language Models via Lens of Dynamic Interactions](https://mlsanity.com/paper/2510.05318/) (2025)
- [STaR-SQL: Self-Taught Reasoner for Text-to-SQL](https://mlsanity.com/paper/2502.13550/) (2025)
- [From Natural Language to SQL: Review of LLM-based Text-to-SQL Systems](https://mlsanity.com/paper/2410.01066/) (2024)
- [ROUTE: Robust Multitask Tuning and Collaboration for Text-to-SQL](https://mlsanity.com/paper/2412.10138/) (2024)
- [Pervasive Annotation Errors Break Text-to-SQL Benchmarks and Leaderboards](https://mlsanity.com/paper/2601.08778/) (2026)
- [Dubo-SQL: Diverse Retrieval-Augmented Generation and Fine Tuning for Text-to-SQL](https://mlsanity.com/paper/2404.12560/) (2024)
- [Uncovering the Impact of Chain-of-Thought Reasoning for Direct Preference Optimization: Lessons from Text-to-SQL](https://mlsanity.com/paper/2502.11656/) (2025)
- [Knapsack Optimization-based Schema Linking for LLM-based Text-to-SQL Generation](https://mlsanity.com/paper/2502.12911/) (2025)
- [ReEx-SQL: Reasoning with Execution-Aware Reinforcement Learning for Text-to-SQL](https://mlsanity.com/paper/2505.12768/) (2025)
- [DAC: Decomposed Automation Correction for Text-to-SQL](https://mlsanity.com/paper/2408.08779/) (2024)
- [Exploring the Landscape of Text-to-SQL with Large Language Models: Progresses, Challenges and Opportunities](https://mlsanity.com/paper/2505.23838/) (2025)
- [Enhancing LLM Fine-tuning for Text-to-SQLs by SQL Quality Measurement](https://mlsanity.com/paper/2410.01869/) (2024)
- [MaskSQL: Safeguarding Privacy for LLM-Based Text-to-SQL via Abstraction](https://mlsanity.com/paper/2509.23459/) (2025)
- [Toward Real-World Table Agents: Capabilities, Workflows, and Design Principles for LLM-based Table Intelligence](https://mlsanity.com/paper/2507.10281/) (2025)
- [Exploring Database Normalization Effects on SQL Generation](https://mlsanity.com/paper/2510.01989/) (2025)
- [AGENTIQL: An Agent-Inspired Multi-Expert Framework for Text-to-SQL Generation](https://mlsanity.com/paper/2510.10661/) (2025)
- [Rethinking Text-to-SQL: Dynamic Multi-turn SQL Interaction for Real-world Database Exploration](https://mlsanity.com/paper/2510.26495/) (2025)
- [ChipXplore: Natural Language Exploration of Hardware Designs and Libraries](https://mlsanity.com/paper/2407.12749/) (2024)

## Abstract (arXiv)

Leading models for the text-to-SQL task heavily rely on proprietary Large Language Models (LLMs), posing concerns over data privacy. Closing the performance gap between small open-source models and large proprietary models is crucial to mitigate this reliance. To this end, we introduce a novel two-stage fine-tuning approach that decomposes the task into two simpler tasks. Through comprehensive evaluation on two large cross-domain datasets and two small LLMs, we show that this approach improves execution accuracy by 3 to 7 percent, effectively aligning the performance of open-source models with their proprietary counterparts.