mlsanityData to 7 Oct 2026

CHASE-SQL: Multi-Path Reasoning and Preference Optimized Candidate Selection in Text-to-SQL

arXiv 2410.01943 ↗First posted 2 Oct 2024cs.LG cs.AI cs.CLcited by 132

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

Abstract (arXiv)

In tackling the challenges of large language model (LLM) performance for Text-to-SQL tasks, we introduce CHASE-SQL, a new framework that employs innovative strategies, using test-time compute in multi-agent modeling to improve candidate generation and selection. CHASE-SQL leverages LLMs' intrinsic knowledge to generate diverse and high-quality SQL candidates using different LLM generators with: (1) a divide-and-conquer method that decomposes complex queries into manageable sub-queries in a single LLM call; (2) chain-of-thought reasoning based on query execution plans, reflecting the steps a database engine takes during execution; and (3) a unique instance-aware synthetic example generation technique, which offers specific few-shot demonstrations tailored to test questions.To identify the best candidate, a selection agent is employed to rank the candidates through pairwise comparisons with a fine-tuned binary-candidates selection LLM. This selection approach has been demonstrated to be more robust over alternatives. The proposed generators-selector framework not only enhances the quality and diversity of SQL queries but also outperforms previous methods. Overall, our proposed CHASE-SQL achieves the state-of-the-art execution accuracy of 73.0% and 73.01% on the test set and development set of the notable BIRD Text-to-SQL dataset benchmark, rendering CHASE-SQL the top submission of the leaderboard (at the time of paper submission).

Key facts. Cited by 132 papers in the map (data through 7 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 38 of its 50 references that are in the map. First posted on arXiv 2 Oct 2024.

Builds on

38 of 50 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024The Death of Schema Linking? Text-to-SQL in the Age of Well-Reasoned Language Models14 Aug

Other references · 37

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  3. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  4. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  5. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  6. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  7. 2022Competition-Level Code Generation with AlphaCodecited by 1,427
  8. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,184
  9. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,120
  10. 2023Teaching Large Language Models to Self-Debugcited by 963
  11. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 667
  12. 2023DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correctioncited by 465
25 more
  1. 2020TAPAS: Weakly Supervised Table Parsing via Pre-trainingcited by 432
  2. 2019RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parserscited by 420
  3. 2020TaBERT: Pretraining for Joint Understanding of Textual and Tabular Datacited by 395
  4. 2023Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluationcited by 378
  5. 2023LEVER: Learning to Verify Language-to-Code Generation with Executioncited by 228
  6. 2024Many-Shot In-Context Learningcited by 203
  7. 2023RESDSQL: Decoupling Schema Linking and Skeleton Parsing for Text-to-SQLcited by 191
  8. 2024CodeS: Towards Building Open-source Language Models for Text-to-SQLcited by 183
  9. 2020GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsingcited by 171
  10. 2024CHESS: Contextual Harnessing for Efficient SQL Synthesiscited by 164
  11. 2023C3: Zero-shot Text-to-SQL with ChatGPTcited by 159
  12. 2023OpenAgents: An Open Platform for Language Agents in the Wildcited by 136
  13. 2021LGESQL: Line Graph Enhanced Text-to-SQL Model with Mixed Local and Non-Local Relationscited by 119
  14. 2024MCS-SQL: Leveraging Multiple Prompts and Multiple-Choice Selection For Text-to-SQL Generationcited by 103
  15. 2023Graphix-T5: Mixing Pre-Trained Transformers with Graph-Aware Layers for Text-to-SQL Parsingcited by 93
  16. 2019A Comprehensive Exploration on WikiSQL with Table-Aware Word Contextualizationcited by 88
  17. 2024The Dawn of Natural Language to SQL: Are We Fully Ready?cited by 78
  18. 2020RYANSQL: Recursively Applying Sketch-based Slot Fillings for Complex Text-to-SQL in Cross-Domain Databasescited by 63
  19. 2024DTS-SQL: Decomposed Text-to-SQL with Small Large Language Modelscited by 45
  20. 2023Enhancing Few-shot Text-to-SQL Capabilities of Large Language Models: A Study on Prompt Design Strategiescited by 44
  21. 2021SADGA: Structure-Aware Dual Graph Aggregation Network for Text-to-SQLcited by 43
  22. 2023SQL-PaLM: Improved Large Language Model Adaptation for Text-to-SQL (extended)cited by 42
  23. 2023Improving Generalization in Language Model-Based Text-to-SQL Semantic Parsing: Two Simple Semantic Boundary-Based Techniquescited by 23
  24. 2024SQL-GEN: Bridging the Dialect Gap for Text-to-SQL Via Synthetic Data And Model Mergingcited by 16
  25. 2022Natural Language Interfaces to Datacited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 76; 2026: 51.

132 papers.

Sort:

By area:

  1. 2025OmniSQL: Synthesizing High-quality Text-to-SQL Data at Scalecited by 90
  2. 2025Reasoning-SQL: Reinforcement Learning with SQL Tailored Partial Rewards for Reasoning-Enhanced Text-to-SQLcited by 59
  3. 2024RSL-SQL: Robust Schema Linking in Text-to-SQL Generationcited by 51
  4. 2024DocETL: Agentic Query Rewriting and Evaluation for Complex Document Processingcited by 49
  5. 2025XiYan-SQL: A Novel Multi-Generator Framework For Text-to-SQLcited by 40
  6. 2025ReFoRCE: A Text-to-SQL Agent with Self-Refinement, Consensus Enforcement, and Column Explorationcited by 34
  7. 2025Is Long Context All You Need? Leveraging LLM's Extended Context for NL2SQLcited by 30
  8. 2025A Survey of Data Agents: Emerging Paradigm or Overstated Hype?cited by 30
  9. 2025LinkAlign: Scalable Schema Linking for Real-World Large-Scale Multi-Database Text-to-SQLcited by 26
  10. 2025Reward-SQL: Boosting Text-to-SQL via Stepwise Execution-Aware Reasoning and Process-Supervised Rewardscited by 25
  11. 2025ExCoT: Optimizing Reasoning for Text-to-SQL with Execution Feedbackcited by 23
  12. 2025From Few to Many: Self-Improving Many-Shot Reasoners Through Iterative Optimization and Generationcited by 20
38 more
  1. 2025Reliable Text-to-SQL with Adaptive Abstentioncited by 19
  2. 2025SQL-o1: A Self-Reward Heuristic Dynamic Search Method for Text-to-SQLcited by 19
  3. 2025BIRD-INTERACT: Re-imagining Text-to-SQL Evaluation for Large Language Models via Lens of Dynamic Interactionscited by 17
  4. 2025CSC-SQL: Corrective Self-Consistency in Text-to-SQL via Reinforcement Learningcited by 15
  5. 2025SHARE: An SLM-based Hierarchical Action CorREction Assistant for Text-to-SQLcited by 15
  6. 2025SWE-SQL: Illuminating LLM Pathways to Solve User SQL Issues in Real-World Applicationscited by 15
  7. 2025AutoLink: Autonomous Schema Exploration and Expansion for Scalable Schema Linking in Text-to-SQL at Scalecited by 15
  8. 2025Understanding, Detecting, and Repairing Real-World In-Context-Learning-Based Text-to-SQL Errorscited by 14
  9. 2025MARS-SQL: A multi-agent reinforcement learning framework for Text-to-SQLcited by 14
  10. 2024ROUTE: Robust Multitask Tuning and Collaboration for Text-to-SQLcited by 13
  11. 2025EllieSQL: Cost-Efficient Text-to-SQL with Complexity-Aware Routingcited by 13
  12. 2025SQLForge: Synthesizing Reliable and Diverse Data to Enhance Text-to-SQL Reasoning in LLMscited by 10
  13. 2025SQLens: An End-to-End Framework for Error Detection and Correction in Text-to-SQLcited by 10
  14. 2025SLM-SQL: An Exploration of Small Language Models for Text-to-SQLcited by 10
  15. 2025When TableQA Meets Noise: A Dual Denoising Framework for Complex Questions and Large-scale Tablescited by 10
  16. 2025MTIR-SQL: Multi-turn Tool-Integrated Reasoning Reinforcement Learning for Text-to-SQLcited by 10
  17. 2024NAT-NL2GQL: A Novel Multi-Agent Framework for Translating Natural Language to Graph Query Languagecited by 9
  18. 2025BASE-SQL: A powerful open source Text-To-SQL baseline approachcited by 9
  19. 2025Uncovering the Impact of Chain-of-Thought Reasoning for Direct Preference Optimization: Lessons from Text-to-SQLcited by 9
  20. 2025Knapsack Optimization-based Schema Linking for LLM-based Text-to-SQL Generationcited by 9
  21. 2025ReEx-SQL: Reasoning with Execution-Aware Reinforcement Learning for Text-to-SQLcited by 8
  22. 2025Cut Costs, Not Accuracy: LLM-Powered Data Processing with Guaranteescited by 8
  23. 2025Text-to-SQL as Dual-State Reasoning: Integrating Adaptive Context and Progressive Generationcited by 8
  24. 2025Graph-Reward-SQL: Execution-Free Reinforcement Learning for Text-to-SQL via Graph Matching and Stepwise Rewardcited by 7
  25. 2026SQL-Trail: Multi-Turn Reinforcement Learning with Interleaved Feedback for Text-to-SQLcited by 7
  26. 2026Query-Aware Flow Diffusion for Graph-Based RAG with Retrieval Guaranteescited by 7
  27. 2025DS-STAR: Data Science Agent for Solving Diverse Tasks across Heterogeneous Formats and Open-Ended Queriescited by 6
  28. 2026Arming Data Agents with Tribal Knowledgecited by 6
  29. 2025Magical: Medical Lay Language Generation via Semantic Invariance and Layperson-tailored Adaptationcited by 5
  30. 2025SQL-of-Thought: Multi-agentic Text-to-SQL with Guided Error Correctioncited by 5
  31. 2025SPFT-SQL: Enhancing Large Language Model for Text-to-SQL Parsing by Self-Play Fine-Tuningcited by 5
  32. 2025SING-SQL: A Synthetic Data Generation Framework for In-Domain Text-to-SQL Translationcited by 5
  33. 2025MTSQL-R1: Towards Long-Horizon Multi-Turn Text-to-SQL via Agentic Trainingcited by 5
  34. 2025Multi-Agent GraphRAG: A Text-to-Cypher Framework for Labeled Property Graphscited by 5
  35. 2026PV-SQL: Synergizing Database Probing and Rule-based Verification for Text-to-SQL Agentscited by 5
  36. 2025MCTS-SQL: Light-Weight LLMs can Master the Text-to-SQL through Monte Carlo Tree Searchcited by 4
  37. 2025Exploring the Landscape of Text-to-SQL with Large Language Models: Progresses, Challenges and Opportunitiescited by 4
  38. 2025RAISE: Reasoning Agent for Interactive SQL Explorationcited by 4

50 most cited of 132 All 132 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025BAPPA: Benchmarking Agents, Plans, and Pipelines for Automated Text-to-SQL Generation
  2. 2025JudgeSQL: Reasoning over SQL Candidates with Weighted Consensus Tournament
  3. 2026Test-Time Verification for Text-to-SQL via Outcome Reward Models
  4. 2026CA-SQL: Complexity-Aware Inference Time Reasoning for Text-to-SQL via Exploration and Compute Budget Allocation
  5. 2023MAC-SQL: A Multi-Agent Collaborative Framework for Text-to-SQL
  6. 2024SelECT-SQL: Self-correcting ensemble Chain-of-Thought for Text-to-SQL
  7. 2024Multi-Turn Interactions for Text-to-SQL with Large Language Models
  8. 2026Integrating Reasoning and Generalization in Text-to-SQL via Self-Enhanced Fine-Tuning