mlsanityData to 14 Oct 2026

ExCoT: Optimizing Reasoning for Text-to-SQL with Execution Feedback

arXiv 2503.19988 ↗First posted 25 Mar 2025cs.LG cs.AI cs.DBcited by 24

On the map: LLM reasoning → Knowledge- and tool-augmented reasoning

also in 1 more
Abstract (arXiv)

Text-to-SQL demands precise reasoning to convert natural language questions into structured queries. While large language models (LLMs) excel in many reasoning tasks, their ability to leverage Chain-of-Thought (CoT) reasoning for text-to-SQL remains underexplored. We identify critical limitations: zero-shot CoT offers minimal gains, and Direct Preference Optimization (DPO) applied without CoT yields marginal improvements. We propose ExCoT, a novel framework that iteratively optimizes open-source LLMs by combining CoT reasoning with off-policy and on-policy DPO, relying solely on execution accuracy as feedback. This approach eliminates the need for reward models or human-annotated preferences. Our experimental results demonstrate significant performance gains: ExCoT improves execution accuracy on BIRD dev set from 57.37% to 68.51% and on Spider test set from 78.81% to 86.59% for LLaMA-3 70B, with Qwen-2.5-Coder demonstrating similar improvements. Our best model achieves state-of-the-art performance in the single-model setting on both BIRD and Spider datasets, notably achieving 68.53% on the BIRD test set.

Key facts. Cited by 24 papers in the map (data through 14 Oct 2026). Placed in Knowledge- and tool-augmented reasoning, one of 17 directions in LLM reasoning. Builds on 23 of its 34 references that are in the map. First posted on arXiv 25 Mar 2025.

Builds on

23 of 34 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024CHASE-SQL: Multi-Path Reasoning and Preference Optimized Candidate Selection in Text-to-SQL2 Oct
  2. 2023ACT-SQL: In-Context Learning for Text-to-SQL with Automatically-Generated Chain-of-Thought26 Oct

Other references · 21

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  4. 2024GPT-4o System Cardcited by 4,588
  5. 2024Qwen2 Technical Reportcited by 2,120
  6. 2024OpenAI o1 System Cardcited by 1,244
  7. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,210
  8. 2024Qwen2.5-Coder Technical Reportcited by 1,203
  9. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,139
  10. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 812
  11. 2019RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parserscited by 424
  12. 2021PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Modelscited by 411
9 more
  1. 2023Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluationcited by 385
  2. 2025rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinkingcited by 313
  3. 2024Direct Language Model Alignment from Online AI Feedbackcited by 260
  4. 2024CHESS: Contextual Harnessing for Efficient SQL Synthesiscited by 169
  5. 2024A Preview of XiYan-SQL: A Multi-Generator Ensemble Framework for Text-to-SQLcited by 46
  6. 2024Curry-DPO: Enhancing Alignment using Curriculum Learning & Ranked Preferencescited by 39
  7. 2021Learning from Executions for Semantic Parsingcited by 7
  8. 2023Automatic Pair Construction for Contrastive Post-trainingcited by 7
  9. 2024Text-to-SQL Calibration: No Need to Ask -- Just Rescale Model Probabilitiescited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 11; 2026: 13.

24 papers.

Sort:

By area:

  1. 2025VerlTool: Towards Holistic Agentic Reinforcement Learning with Tool Usecited by 96
  2. 2025Arctic-Text2SQL-R1: Simple Rewards, Strong Reasoning in Text-to-SQLcited by 38
  3. 2025Reward-SQL: Boosting Text-to-SQL via Stepwise Execution-Aware Reasoning and Process-Supervised Rewardscited by 27
  4. 2025SWE-SQL: Illuminating LLM Pathways to Solve User SQL Issues in Real-World Applicationscited by 16
  5. 2025Think2SQL: Reinforce LLM Reasoning Capabilities for Text2SQLcited by 14
  6. 2025MTIR-SQL: Multi-turn Tool-Integrated Reasoning Reinforcement Learning for Text-to-SQLcited by 10
  7. 2025ReEx-SQL: Reasoning with Execution-Aware Reinforcement Learning for Text-to-SQLcited by 8
  8. 2025Rethinking Schema Linking: A Context-Aware Bidirectional Retrieval Approach for Text-to-SQLcited by 4
  9. 2025SteinerSQL: Graph-Guided Mathematical Reasoning for Text-to-SQL Generationcited by 1
  10. 2025HES-SQL: Hybrid Reasoning for Efficient Text-to-SQL with Structural Skeleton Guidancecited by 1
  11. 2025Knowledge Distillation with Structured Chain-of-Thought for Text-to-SQLcited by 1
  12. 2026EvolSQL: Structure-Aware Evolution for Scalable Text-to-SQL Data Synthesiscited by 1
12 more
  1. 2026stratum: A System Infrastructure for Massive Agent-Centric ML Workloadscited by 1
  2. 2026Human-Level Text-to-SQL via Reinforcement Learning on Verified Data, Without Pipeline Engineeringcited by 1
  3. 2026EXPO-SQL: Execution-based Clause-level Policy Optimization for Text-to-SQLcited by 1
  4. 2026Boundary-Aware NL2SQL: Integrating Reliability through Hybrid Reward and Data Synthesiscited by 0
  5. 2026Natural Language Interfaces for Spatial and Temporal Databases: A Comprehensive Overview of Methods, Taxonomy, and Future Directionscited by 0
  6. 2026Reliable Answers for Recurring Questions: Boosting Text-to-SQL Accuracy with Template Constrained Decodingcited by 0
  7. 2026FINER-SQL: Boosting Small Language Models for Text-to-SQLcited by 0
  8. 2026Residual Skill Optimization for Text-to-SQL Ensemblescited by 0
  9. 2026Integrating Reasoning and Generalization in Text-to-SQL via Self-Enhanced Fine-Tuningcited by 0
  10. 2026DecoSearch: Complexity-Aware Routing and Plan-Level Repair for Text-to-SQLcited by 0
  11. 2026What Drives Recovery in Agentic Text-to-Cypher? LAST-CQ: An LLM Agent Self-Refinement Frameworkcited by 0
  12. 2026AptMQL-Bench: From Text-to-SQL to Text-to-MQL via Access-Pattern Schema Design and Data-Preserving Migrationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Uncovering the Impact of Chain-of-Thought Reasoning for Direct Preference Optimization: Lessons from Text-to-SQL
  2. 2026Learning When to Reason for Text-to-SQL via SFT and DPO
  3. 2024Lucy: Think and Reason to Solve Text-to-SQL
  4. 2025SQLForge: Synthesizing Reliable and Diverse Data to Enhance Text-to-SQL Reasoning in LLMs
  5. 2025JudgeSQL: Reasoning over SQL Candidates with Weighted Consensus Tournament
  6. 2026IESR:Efficient MCTS-Based Modular Reasoning for Text-to-SQL with Large Language Models
  7. 2025Sparks of Tabular Reasoning via Text2SQL Reinforcement Learning
  8. 2025Rationalization Models for Text-to-SQL