mlsanityData to 14 Oct 2026

Exploring Chain-of-Thought Style Prompting for Text-to-SQL

arXiv 2305.14215 ↗First posted 23 May 2023cs.CLcited by 69

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

In-context learning with large language models (LLMs) has recently caught increasing attention due to its superior few-shot performance on various tasks. However, its performance on text-to-SQL parsing still has much room for improvement. In this paper, we hypothesize that a crucial aspect of LLMs to improve for text-to-SQL parsing is their multi-step reasoning ability. Thus, we systematically study how to enhance LLMs' reasoning ability through chain of thought (CoT) style prompting, including the original chain-of-thought prompting (Wei et al., 2022b) and least-to-most prompting (Zhou et al., 2023). Our experiments demonstrate that iterative prompting as in Zhou et al. (2023) may be unnecessary for text-to-SQL parsing, and using detailed reasoning steps tends to have more error propagation issues. Based on these findings, we propose a new CoT-style prompting method for text-to-SQL parsing. It brings 5.2 and 6.5 point absolute gains on the Spider development set and the Spider Realistic set, respectively, compared to the standard prompting method without reasoning steps; 2.4 and 1.5 point absolute gains, compared to the least-to-most prompting method.

Key facts. Cited by 69 papers in the map (data through 14 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 34 of its 50 references that are in the map. First posted on arXiv 23 May 2023.

Builds on

34 of 50 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022Large Language Models are Zero-Shot Reasoners24 May
  2. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Models21 May

Other references · 32

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  3. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  4. 2022Emergent Abilities of Large Language Modelscited by 2,512
  5. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  6. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,139
  7. 2023DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correctioncited by 477
  8. 2019RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parserscited by 424
  9. 2021PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Modelscited by 411
  10. 2022Selective Annotation Makes Language Models Better Few-Shot Learnerscited by 243
  11. 2023LEVER: Learning to Verify Language-to-Code Generation with Executioncited by 239
  12. 2020GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsingcited by 171
20 more
  1. 2020Compositional Generalization and Natural Language Variation: Can a Semantic Parsing Approach Handle Both?cited by 147
  2. 2020Break It Down: A Question Understanding Benchmarkcited by 131
  3. 2020Semantic Evaluation for Text-to-SQL with Distilled Test Suitescited by 126
  4. 2020Structure-Grounded Pretraining for Text-to-SQLcited by 126
  5. 2019Multi-hop Reading Comprehension through Question Decomposition and Rescoringcited by 119
  6. 2023A comprehensive evaluation of ChatGPT's zero-shot Text-to-SQL capabilitycited by 119
  7. 2022Evaluating the Text-to-SQL Capabilities of Large Language Modelscited by 101
  8. 2017Learning a Neural Semantic Parser from User Feedbackcited by 99
  9. 2022RASAT: Integrating Relational Structures into Pretrained Seq2Seq Model for Text-to-SQLcited by 85
  10. 2023Dr.Spider: A Diagnostic Evaluation Benchmark towards Text-to-SQL Robustnesscited by 53
  11. 2020Speak to your Parser: Interactive Text-to-SQL with Natural Language Feedbackcited by 44
  12. 2019Model-based Interactive Semantic Parsing: A Unified Framework and A Text-to-SQL Case Studycited by 40
  13. 2021NL-EDIT: Correcting semantic parse errors through natural language interactioncited by 33
  14. 2020Photon: A Robust Cross-Domain Text-to-SQL Systemcited by 26
  15. 2020An Imitation Game for Learning Semantic Parsers from User Interactioncited by 23
  16. 2023Text-to-SQL Error Correction with Language Models of Codecited by 21
  17. 2021Weakly Supervised Text-to-SQL Parsing through Question Decompositioncited by 15
  18. 2018Neural Compositional Denotational Semantics for Question Answeringcited by 11
  19. 2020"What Do You Mean by That?" A Parser-Independent Interactive Approach for Enhancing Text-to-SQLcited by 11
  20. 2021Towards Transparent Interactive Semantic Parsing via Step-by-Step Correctioncited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 6; 2024: 16; 2025: 33; 2026: 14.

69 papers.

Sort:

By area:

  1. 2023MAC-SQL: A Multi-Agent Collaborative Framework for Text-to-SQLcited by 164
  2. 2024TableBench: A Comprehensive and Complex Benchmark for Table Question Answeringcited by 128
  3. 2024Next-Generation Database Interfaces: A Survey of LLM-based Text-to-SQLcited by 120
  4. 2024MCS-SQL: Leveraging Multiple Prompts and Multiple-Choice Selection For Text-to-SQL Generationcited by 105
  5. 2024A Survey on Employing Large Language Models for Text-to-SQL Taskscited by 67
  6. 2024A Survey of Text-to-SQL in the Era of LLMs: Where are we, and where are we going?cited by 56
  7. 2024Synthesizing Text-to-SQL Data from Weak and Strong LLMscited by 54
  8. 2025OpenSearch-SQL: Enhancing Text-to-SQL with Dynamic Few-shot and Consistency Alignmentcited by 53
  9. 2024RSL-SQL: Robust Schema Linking in Text-to-SQL Generationcited by 52
  10. 2023SQL-PaLM: Improved Large Language Model Adaptation for Text-to-SQL (extended)cited by 42
  11. 2024Middleware for LLMs: Tools Are Instrumental for Language Agents in Complex Environmentscited by 39
  12. 2024TabSQLify: Enhancing Reasoning Capabilities of LLMs Through Table Decompositioncited by 39
38 more
  1. 2025Arctic-Text2SQL-R1: Simple Rewards, Strong Reasoning in Text-to-SQLcited by 38
  2. 2024PURPLE: Making a Large Language Model a Better SQL Writercited by 34
  3. 2024Decomposition for Enhancing Attention: Improving LLM-based Text-to-SQL through Workflow Paradigmcited by 31
  4. 2025Is Long Context All You Need? Leveraging LLM's Extended Context for NL2SQLcited by 30
  5. 2023Natural Language Interfaces for Tabular Data Querying and Visualization: A Surveycited by 27
  6. 2024MAG-SQL: Multi-Agent Generative Approach with Soft Schema Linking and Iterative Sub-SQL Refinement for Text-to-SQLcited by 22
  7. 2023Selective Demonstrations for Cross-domain Text-to-SQLcited by 21
  8. 2025Reliable Text-to-SQL with Adaptive Abstentioncited by 20
  9. 2025SQL-o1: A Self-Reward Heuristic Dynamic Search Method for Text-to-SQLcited by 19
  10. 2025RAG over Tables: Hierarchical Memory Index, Multi-Stage Retrieval, and Benchmarkingcited by 18
  11. 2025MARS-SQL: A multi-agent reinforcement learning framework for Text-to-SQLcited by 15
  12. 2024ROUTE: Robust Multitask Tuning and Collaboration for Text-to-SQLcited by 13
  13. 2025An LLM-Based Approach for Insight Generation in Data Analysiscited by 12
  14. 2025Text2SQL-Flow: A Robust SQL-Aware Data Augmentation Framework for Text-to-SQLcited by 11
  15. 2025Knapsack Optimization-based Schema Linking for LLM-based Text-to-SQL Generationcited by 9
  16. 2025Knowledge Base Construction for Knowledge-Augmented Text-to-SQLcited by 9
  17. 2023R3-NL2GQL: A Model Coordination and Knowledge Graph Alignment Approach for NL2GQLcited by 7
  18. 2024LG AI Research & KAIST at EHRSQL 2024: Self-Training Large Language Models with Pseudo-Labeled Unanswerable Questions for a Reliable Text-to-SQL System on EHRscited by 7
  19. 2025Dialogue Benchmark Generation from Knowledge Graphs with Cost-Effective Retrieval-Augmented LLMscited by 7
  20. 2025Graph-Reward-SQL: Execution-Free Reinforcement Learning for Text-to-SQL via Graph Matching and Stepwise Rewardcited by 7
  21. 2025What Makes a Good Natural Language Prompt?cited by 7
  22. 2023Battle of the Large Language Models: Dolly vs LLaMA vs Vicuna vs Guanaco vs Bard vs ChatGPT -- A Text-to-SQL Parsing Comparisoncited by 6
  23. 2025Chain-of-Query: Unleashing the Power of LLMs in SQL-Aided Table Understanding via Multi-Agent Collaborationcited by 6
  24. 2025SQL-of-Thought: Multi-agentic Text-to-SQL with Guided Error Correctioncited by 6
  25. 2025Automatic database description generation for Text-to-SQLcited by 5
  26. 2025DB-Explore: Automated Database Exploration and Instruction Synthesis for Text-to-SQLcited by 5
  27. 2025Calibrating LLMs for Text-to-SQL Parsing by Leveraging Sub-clause Frequenciescited by 5
  28. 2026EvoSchema: Towards Text-to-SQL Robustness Against Schema Evolutioncited by 5
  29. 2024Tool-Assisted Agent on SQL Inspection and Refinement in Real-World Scenarioscited by 4
  30. 2025GateLens: A Reasoning-Enhanced LLM Agent for Automotive Software Release Analyticscited by 4
  31. 2025Exploring the Landscape of Text-to-SQL with Large Language Models: Progresses, Challenges and Opportunitiescited by 4
  32. 2025MultiTEND: A Multilingual Benchmark for Natural Language to NoSQL Query Translationcited by 3
  33. 2025SDE-SQL: Enhancing Text-to-SQL Generation in Large Language Models via Self-Driven Exploration with SQL Probescited by 2
  34. 2025Toward Real-World Table Agents: Capabilities, Workflows, and Design Principles for LLM-based Table Intelligencecited by 2
  35. 2025AGENTIQL: An Agent-Inspired Multi-Expert Framework for Text-to-SQL Generationcited by 2
  36. 2025From Questions to Queries: An AI-powered Multi-Agent Framework for Spatial Text-to-SQLcited by 2
  37. 2026ErrorLLM: Modeling SQL Errors for Text-to-SQL Refinementcited by 2
  38. 2024KeyInst: Keyword Instruction for Improving SQL Formulation in Text-to-SQLcited by 1

50 most cited of 69 All 69 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Divide and Prompt: Chain of Thought Prompting for Text-to-SQL
  2. 2023ACT-SQL: In-Context Learning for Text-to-SQL with Automatically-Generated Chain-of-Thought
  3. 2023How to Prompt LLMs for Text-to-SQL: A Study in Zero-shot, Single-domain, and Cross-domain Settings
  4. 2022Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters
  5. 2024Chain-of-Thought Reasoning Without Prompting
  6. 2024RB-SQL: A Retrieval-based LLM Framework for Text-to-SQL
  7. 2024Enhancing Chain of Thought Prompting in Large Language Models via Reasoning Patterns
  8. 2023Chain-Of-Thought Prompting Under Streaming Batch: A Case Study