# CHASE-SQL: Multi-Path Reasoning and Preference Optimized Candidate Selection in Text-to-SQL

arXiv: https://arxiv.org/abs/2410.01943 · HTML: https://mlsanity.com/paper/2410.01943/

## Key facts

- Cited by 138 papers in the map (data through 14 Oct 2026).
- Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning.
- Builds on 38 of its 50 references that are in the map.
- First posted on arXiv 2 Oct 2024.

## Placed in

- LLM reasoning / [Knowledge- and tool-augmented reasoning](https://mlsanity.com/lr/knowledge-and-tool-augmented-reasoning/)

## Builds on (most cited first, 30 of 38)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Self-Instruct: Aligning Language Models with Self-Generated Instructions](https://mlsanity.com/paper/2212.10560/) (2022)
- [Competition-Level Code Generation with AlphaCode](https://mlsanity.com/paper/2203.07814/) (2022)
- [What Makes Good In-Context Examples for GPT-3?](https://mlsanity.com/paper/2101.06804/) (2021)
- [Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task](https://mlsanity.com/paper/1809.08887/) (2018)
- [Teaching Large Language Models to Self-Debug](https://mlsanity.com/paper/2304.05128/) (2023)
- [Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs](https://mlsanity.com/paper/2305.03111/) (2023)
- [DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correction](https://mlsanity.com/paper/2304.11015/) (2023)
- [TAPAS: Weakly Supervised Table Parsing via Pre-training](https://mlsanity.com/paper/2004.02349/) (2020)
- [RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers](https://mlsanity.com/paper/1911.04942/) (2019)
- [TaBERT: Pretraining for Joint Understanding of Textual and Tabular Data](https://mlsanity.com/paper/2005.08314/) (2020)
- [Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluation](https://mlsanity.com/paper/2308.15363/) (2023)
- [LEVER: Learning to Verify Language-to-Code Generation with Execution](https://mlsanity.com/paper/2302.08468/) (2023)
- [Many-Shot In-Context Learning](https://mlsanity.com/paper/2404.11018/) (2024)
- [RESDSQL: Decoupling Schema Linking and Skeleton Parsing for Text-to-SQL](https://mlsanity.com/paper/2302.05965/) (2023)
- [CodeS: Towards Building Open-source Language Models for Text-to-SQL](https://mlsanity.com/paper/2402.16347/) (2024)
- [GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsing](https://mlsanity.com/paper/2009.13845/) (2020)
- [CHESS: Contextual Harnessing for Efficient SQL Synthesis](https://mlsanity.com/paper/2405.16755/) (2024)
- [C3: Zero-shot Text-to-SQL with ChatGPT](https://mlsanity.com/paper/2307.07306/) (2023)
- [OpenAgents: An Open Platform for Language Agents in the Wild](https://mlsanity.com/paper/2310.10634/) (2023)
- [LGESQL: Line Graph Enhanced Text-to-SQL Model with Mixed Local and Non-Local Relations](https://mlsanity.com/paper/2106.01093/) (2021)
- [MCS-SQL: Leveraging Multiple Prompts and Multiple-Choice Selection For Text-to-SQL Generation](https://mlsanity.com/paper/2405.07467/) (2024)
- [Graphix-T5: Mixing Pre-Trained Transformers with Graph-Aware Layers for Text-to-SQL Parsing](https://mlsanity.com/paper/2301.07507/) (2023)
- [A Comprehensive Exploration on WikiSQL with Table-Aware Word Contextualization](https://mlsanity.com/paper/1902.01069/) (2019)
- [The Dawn of Natural Language to SQL: Are We Fully Ready?](https://mlsanity.com/paper/2406.01265/) (2024)
- [The Death of Schema Linking? Text-to-SQL in the Age of Well-Reasoned Language Models](https://mlsanity.com/paper/2408.07702/) (2024)

## Built on by (most cited first, 30 of 138)

- [OmniSQL: Synthesizing High-quality Text-to-SQL Data at Scale](https://mlsanity.com/paper/2503.02240/) (2025)
- [Reasoning-SQL: Reinforcement Learning with SQL Tailored Partial Rewards for Reasoning-Enhanced Text-to-SQL](https://mlsanity.com/paper/2503.23157/) (2025)
- [DocETL: Agentic Query Rewriting and Evaluation for Complex Document Processing](https://mlsanity.com/paper/2410.12189/) (2024)
- [RSL-SQL: Robust Schema Linking in Text-to-SQL Generation](https://mlsanity.com/paper/2411.00073/) (2024)
- [XiYan-SQL: A Novel Multi-Generator Framework For Text-to-SQL](https://mlsanity.com/paper/2507.04701/) (2025)
- [ReFoRCE: A Text-to-SQL Agent with Self-Refinement, Consensus Enforcement, and Column Exploration](https://mlsanity.com/paper/2502.00675/) (2025)
- [Is Long Context All You Need? Leveraging LLM's Extended Context for NL2SQL](https://mlsanity.com/paper/2501.12372/) (2025)
- [A Survey of Data Agents: Emerging Paradigm or Overstated Hype?](https://mlsanity.com/paper/2510.23587/) (2025)
- [LinkAlign: Scalable Schema Linking for Real-World Large-Scale Multi-Database Text-to-SQL](https://mlsanity.com/paper/2503.18596/) (2025)
- [Reward-SQL: Boosting Text-to-SQL via Stepwise Execution-Aware Reasoning and Process-Supervised Rewards](https://mlsanity.com/paper/2505.04671/) (2025)
- [ExCoT: Optimizing Reasoning for Text-to-SQL with Execution Feedback](https://mlsanity.com/paper/2503.19988/) (2025)
- [From Few to Many: Self-Improving Many-Shot Reasoners Through Iterative Optimization and Generation](https://mlsanity.com/paper/2502.00330/) (2025)
- [Reliable Text-to-SQL with Adaptive Abstention](https://mlsanity.com/paper/2501.10858/) (2025)
- [SQL-o1: A Self-Reward Heuristic Dynamic Search Method for Text-to-SQL](https://mlsanity.com/paper/2502.11741/) (2025)
- [BIRD-INTERACT: Re-imagining Text-to-SQL Evaluation for Large Language Models via Lens of Dynamic Interactions](https://mlsanity.com/paper/2510.05318/) (2025)
- [AutoLink: Autonomous Schema Exploration and Expansion for Scalable Schema Linking in Text-to-SQL at Scale](https://mlsanity.com/paper/2511.17190/) (2025)
- [SHARE: An SLM-based Hierarchical Action CorREction Assistant for Text-to-SQL](https://mlsanity.com/paper/2506.00391/) (2025)
- [SWE-SQL: Illuminating LLM Pathways to Solve User SQL Issues in Real-World Applications](https://mlsanity.com/paper/2506.18951/) (2025)
- [CSC-SQL: Corrective Self-Consistency in Text-to-SQL via Reinforcement Learning](https://mlsanity.com/paper/2505.13271/) (2025)
- [MARS-SQL: A multi-agent reinforcement learning framework for Text-to-SQL](https://mlsanity.com/paper/2511.01008/) (2025)
- [Understanding, Detecting, and Repairing Real-World In-Context-Learning-Based Text-to-SQL Errors](https://mlsanity.com/paper/2501.09310/) (2025)
- [ROUTE: Robust Multitask Tuning and Collaboration for Text-to-SQL](https://mlsanity.com/paper/2412.10138/) (2024)
- [EllieSQL: Cost-Efficient Text-to-SQL with Complexity-Aware Routing](https://mlsanity.com/paper/2503.22402/) (2025)
- [SQLens: An End-to-End Framework for Error Detection and Correction in Text-to-SQL](https://mlsanity.com/paper/2506.04494/) (2025)
- [SQLForge: Synthesizing Reliable and Diverse Data to Enhance Text-to-SQL Reasoning in LLMs](https://mlsanity.com/paper/2505.13725/) (2025)
- [SLM-SQL: An Exploration of Small Language Models for Text-to-SQL](https://mlsanity.com/paper/2507.22478/) (2025)
- [When TableQA Meets Noise: A Dual Denoising Framework for Complex Questions and Large-scale Tables](https://mlsanity.com/paper/2509.17680/) (2025)
- [MTIR-SQL: Multi-turn Tool-Integrated Reasoning Reinforcement Learning for Text-to-SQL](https://mlsanity.com/paper/2510.25510/) (2025)
- [Text-to-SQL as Dual-State Reasoning: Integrating Adaptive Context and Progressive Generation](https://mlsanity.com/paper/2511.21402/) (2025)
- [NAT-NL2GQL: A Novel Multi-Agent Framework for Translating Natural Language to Graph Query Language](https://mlsanity.com/paper/2412.10434/) (2024)

## Abstract (arXiv)

In tackling the challenges of large language model (LLM) performance for Text-to-SQL tasks, we introduce CHASE-SQL, a new framework that employs innovative strategies, using test-time compute in multi-agent modeling to improve candidate generation and selection. CHASE-SQL leverages LLMs' intrinsic knowledge to generate diverse and high-quality SQL candidates using different LLM generators with: (1) a divide-and-conquer method that decomposes complex queries into manageable sub-queries in a single LLM call; (2) chain-of-thought reasoning based on query execution plans, reflecting the steps a database engine takes during execution; and (3) a unique instance-aware synthetic example generation technique, which offers specific few-shot demonstrations tailored to test questions.To identify the best candidate, a selection agent is employed to rank the candidates through pairwise comparisons with a fine-tuned binary-candidates selection LLM. This selection approach has been demonstrated to be more robust over alternatives. The proposed generators-selector framework not only enhances the quality and diversity of SQL queries but also outperforms previous methods. Overall, our proposed CHASE-SQL achieves the state-of-the-art execution accuracy of 73.0% and 73.01% on the test set and development set of the notable BIRD Text-to-SQL dataset benchmark, rendering CHASE-SQL the top submission of the leaderboard (at the time of paper submission).