# Understanding, Detecting, and Repairing Real-World In-Context-Learning-Based Text-to-SQL Errors

arXiv: https://arxiv.org/abs/2501.09310 · HTML: https://mlsanity.com/paper/2501.09310/

## Key facts

- Cited by 14 papers in the map (data through 7 Oct 2026).
- Builds on 57 of its 81 references that are in the map.
- First posted on arXiv 16 Jan 2025.

## Builds on (most cited first, 30 of 57)

- [Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task](https://mlsanity.com/paper/1809.08887/) (2018)
- [Large Language Models Cannot Self-Correct Reasoning Yet](https://mlsanity.com/paper/2310.01798/) (2023)
- [Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning](https://mlsanity.com/paper/1709.00103/) (2017)
- [Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs](https://mlsanity.com/paper/2305.03111/) (2023)
- [DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correction](https://mlsanity.com/paper/2304.11015/) (2023)
- [Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluation](https://mlsanity.com/paper/2308.15363/) (2023)
- [UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models](https://mlsanity.com/paper/2201.05966/) (2022)
- [Towards Complex Text-to-SQL in Cross-Domain Database with Intermediate Representation](https://mlsanity.com/paper/1905.08205/) (2019)
- [RESDSQL: Decoupling Schema Linking and Skeleton Parsing for Text-to-SQL](https://mlsanity.com/paper/2302.05965/) (2023)
- [CodeS: Towards Building Open-source Language Models for Text-to-SQL](https://mlsanity.com/paper/2402.16347/) (2024)
- [GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsing](https://mlsanity.com/paper/2009.13845/) (2020)
- [C3: Zero-shot Text-to-SQL with ChatGPT](https://mlsanity.com/paper/2307.07306/) (2023)
- [MAC-SQL: A Multi-Agent Collaborative Framework for Text-to-SQL](https://mlsanity.com/paper/2312.11242/) (2023)
- [SQLNet: Generating Structured Queries From Natural Language Without Reinforcement Learning](https://mlsanity.com/paper/1711.04436/) (2017)
- [Bridging Textual and Tabular Data for Cross-Domain Text-to-SQL Semantic Parsing](https://mlsanity.com/paper/2012.12627/) (2020)
- [CHASE-SQL: Multi-Path Reasoning and Preference Optimized Candidate Selection in Text-to-SQL](https://mlsanity.com/paper/2410.01943/) (2024)
- [Semantic Evaluation for Text-to-SQL with Distilled Test Suites](https://mlsanity.com/paper/2010.02840/) (2020)
- [Coarse-to-Fine Decoding for Neural Semantic Parsing](https://mlsanity.com/paper/1805.04793/) (2018)
- [MCS-SQL: Leveraging Multiple Prompts and Multiple-Choice Selection For Text-to-SQL Generation](https://mlsanity.com/paper/2405.07467/) (2024)
- [SmBoP: Semi-autoregressive Bottom-up Semantic Parsing](https://mlsanity.com/paper/2010.12412/) (2020)
- [Graphix-T5: Mixing Pre-Trained Transformers with Graph-Aware Layers for Text-to-SQL Parsing](https://mlsanity.com/paper/2301.07507/) (2023)
- [A Comprehensive Exploration on WikiSQL with Table-Aware Word Contextualization](https://mlsanity.com/paper/1902.01069/) (2019)
- [TypeSQL: Knowledge-based Type-Aware Neural Text-to-SQL Generation](https://mlsanity.com/paper/1804.09769/) (2018)
- [The Dawn of Natural Language to SQL: Are We Fully Ready?](https://mlsanity.com/paper/2406.01265/) (2024)
- [SyntaxSQLNet: Syntax Tree Networks for Complex and Cross-DomainText-to-SQL Task](https://mlsanity.com/paper/1810.05237/) (2018)
- [The Death of Schema Linking? Text-to-SQL in the Age of Well-Reasoned Language Models](https://mlsanity.com/paper/2408.07702/) (2024)
- [Natural SQL: Making SQL Easier to Infer from Natural Language Specifications](https://mlsanity.com/paper/2109.05153/) (2021)
- [ACT-SQL: In-Context Learning for Text-to-SQL with Automatically-Generated Chain-of-Thought](https://mlsanity.com/paper/2310.17342/) (2023)
- [RSL-SQL: Robust Schema Linking in Text-to-SQL Generation](https://mlsanity.com/paper/2411.00073/) (2024)
- [Before Generation, Align it! A Novel and Effective Strategy for Mitigating Hallucinations in Text-to-SQL Generation](https://mlsanity.com/paper/2405.15307/) (2024)

## Built on by (most cited first, 14 of 14)

- [ReEx-SQL: Reasoning with Execution-Aware Reinforcement Learning for Text-to-SQL](https://mlsanity.com/paper/2505.12768/) (2025)
- [Query and Conquer: Execution-Guided SQL Generation](https://mlsanity.com/paper/2503.24364/) (2025)
- [SQL-of-Thought: Multi-agentic Text-to-SQL with Guided Error Correction](https://mlsanity.com/paper/2509.00581/) (2025)
- [PV-SQL: Synergizing Database Probing and Rule-based Verification for Text-to-SQL Agents](https://mlsanity.com/paper/2604.17653/) (2026)
- [Chain-of-Query: Unleashing the Power of LLMs in SQL-Aided Table Understanding via Multi-Agent Collaboration](https://mlsanity.com/paper/2508.15809/) (2025)
- [Semantic Layers for Reliable LLM-Powered Data Analytics: A Paired Benchmark of Accuracy and Hallucination Across Three Frontier Models](https://mlsanity.com/paper/2604.25149/) (2026)
- [SDE-SQL: Enhancing Text-to-SQL Generation in Large Language Models via Self-Driven Exploration with SQL Probes](https://mlsanity.com/paper/2506.07245/) (2025)
- [Node-Level Uncertainty Estimation in LLM-Generated SQL](https://mlsanity.com/paper/2511.13984/) (2025)
- [GBV-SQL: Guided Generation and SQL2Text Back-Translation Validation for Multi-Agent Text2SQL](https://mlsanity.com/paper/2509.12612/) (2025)
- [SteinerSQL: Graph-Guided Mathematical Reasoning for Text-to-SQL Generation](https://mlsanity.com/paper/2509.19623/) (2025)
- [Both Ends Count! Just How Good are LLM Agents at "Text-to-Big SQL"?](https://mlsanity.com/paper/2602.21480/) (2026)
- [Leveraging RAG-LLMs for Urban Mobility Simulation and Analysis](https://mlsanity.com/paper/2507.10382/) (2025)
- [SemanticAgent: A Semantics-Aware Framework for Text-to-SQL Data Synthesis](https://mlsanity.com/paper/2604.21414/) (2026)
- [Causal Episodic Memory for Feedback-Driven Agent Repair](https://mlsanity.com/paper/2608.05906/) (2026)

## Abstract (arXiv)

Large language models (LLMs) have been adopted for text-to-SQL tasks, utilizing their in-context learning (ICL) capability to translate natural language questions into SQL queries. However, such a technique faces correctness problems. In this paper, we conduct the first comprehensive study of text-to-SQL errors of ICL-based techniques. Our study covers four representative ICL-based techniques, five basic repairing methods, two benchmarks, and two LLM settings. We find that text-to-SQL errors are widespread and summarize 27 error types of 7 categories. We also find that existing repairing attempts have limited correctness improvement while having high computational overhead and many mis-repairs. Based on these findings, we propose MapleDoctor, a novel text-to-SQL error detection and repairing framework. The evaluation demonstrates that MapleDoctor outperforms existing solutions by repairing 13.8% more queries with a negligible number of mis-repairs and reducing 67.4% repair latency. The artifact is publicly available at GitHub.