mlsanityData to 7 Oct 2026

Understanding, Detecting, and Repairing Real-World In-Context-Learning-Based Text-to-SQL Errors

arXiv 2501.09310 ↗First posted 16 Jan 2025cs.CL cs.AI cs.SEcited by 14

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) have been adopted for text-to-SQL tasks, utilizing their in-context learning (ICL) capability to translate natural language questions into SQL queries. However, such a technique faces correctness problems. In this paper, we conduct the first comprehensive study of text-to-SQL errors of ICL-based techniques. Our study covers four representative ICL-based techniques, five basic repairing methods, two benchmarks, and two LLM settings. We find that text-to-SQL errors are widespread and summarize 27 error types of 7 categories. We also find that existing repairing attempts have limited correctness improvement while having high computational overhead and many mis-repairs. Based on these findings, we propose MapleDoctor, a novel text-to-SQL error detection and repairing framework. The evaluation demonstrates that MapleDoctor outperforms existing solutions by repairing 13.8% more queries with a negligible number of mis-repairs and reducing 67.4% repair latency. The artifact is publicly available at GitHub.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 57 of its 81 references that are in the map. First posted on arXiv 16 Jan 2025.

Builds on

57 of 81 references.

References · 57

  1. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,120
  2. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 965
  3. 2017Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learningcited by 679
  4. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 667
  5. 2023DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correctioncited by 465
  6. 2023Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluationcited by 378
  7. 2022UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Modelscited by 231
  8. 2019Towards Complex Text-to-SQL in Cross-Domain Database with Intermediate Representationcited by 210
  9. 2023RESDSQL: Decoupling Schema Linking and Skeleton Parsing for Text-to-SQLcited by 191
  10. 2024CodeS: Towards Building Open-source Language Models for Text-to-SQLcited by 183
  11. 2020GraPPa: Grammar-Augmented Pre-Training for Table Semantic Parsingcited by 171
  12. 2023C3: Zero-shot Text-to-SQL with ChatGPTcited by 159
38 more
  1. 2023MAC-SQL: A Multi-Agent Collaborative Framework for Text-to-SQLcited by 158
  2. 2017SQLNet: Generating Structured Queries From Natural Language Without Reinforcement Learningcited by 135
  3. 2020Bridging Textual and Tabular Data for Cross-Domain Text-to-SQL Semantic Parsingcited by 133
  4. 2024CHASE-SQL: Multi-Path Reasoning and Preference Optimized Candidate Selection in Text-to-SQLcited by 132
  5. 2020Semantic Evaluation for Text-to-SQL with Distilled Test Suitescited by 123
  6. 2018Coarse-to-Fine Decoding for Neural Semantic Parsingcited by 110
  7. 2024MCS-SQL: Leveraging Multiple Prompts and Multiple-Choice Selection For Text-to-SQL Generationcited by 103
  8. 2020SmBoP: Semi-autoregressive Bottom-up Semantic Parsingcited by 97
  9. 2023Graphix-T5: Mixing Pre-Trained Transformers with Graph-Aware Layers for Text-to-SQL Parsingcited by 93
  10. 2019A Comprehensive Exploration on WikiSQL with Table-Aware Word Contextualizationcited by 88
  11. 2018TypeSQL: Knowledge-based Type-Aware Neural Text-to-SQL Generationcited by 81
  12. 2024The Dawn of Natural Language to SQL: Are We Fully Ready?cited by 78
  13. 2018SyntaxSQLNet: Syntax Tree Networks for Complex and Cross-DomainText-to-SQL Taskcited by 74
  14. 2024The Death of Schema Linking? Text-to-SQL in the Age of Well-Reasoned Language Modelscited by 67
  15. 2021Natural SQL: Making SQL Easier to Infer from Natural Language Specificationscited by 66
  16. 2023ACT-SQL: In-Context Learning for Text-to-SQL with Automatically-Generated Chain-of-Thoughtcited by 65
  17. 2024RSL-SQL: Robust Schema Linking in Text-to-SQL Generationcited by 51
  18. 2024Before Generation, Align it! A Novel and Effective Strategy for Mitigating Hallucinations in Text-to-SQL Generationcited by 49
  19. 2023SQL-PaLM: Improved Large Language Model Adaptation for Text-to-SQL (extended)cited by 42
  20. 2023ScienceBenchmark: A Complex Real-World Benchmark for Evaluating Natural Language to SQL Systemscited by 42
  21. 2024FinSQL: Model-Agnostic LLMs-based Text-to-SQL Framework for Financial Analysiscited by 35
  22. 2024E-SQL: Direct Schema Linking via Question Enrichment in Text-to-SQLcited by 35
  23. 2024PURPLE: Making a Large Language Model a Better SQL Writercited by 34
  24. 2024MAGIC: Generating Self-Correction Guideline for In-Context Text-to-SQLcited by 34
  25. 2018Natural Language to Structured Query Generation via Meta-Learningcited by 33
  26. 2020Hybrid Ranking Network for Text-to-SQLcited by 33
  27. 2024Understanding the Effects of Noise in Text-to-SQL: An Examination of the BIRD-Bench Benchmarkcited by 33
  28. 2023Benchmarking and Improving Text-to-SQL Generation under Ambiguitycited by 31
  29. 2024Decomposition for Enhancing Attention: Improving LLM-based Text-to-SQL through Workflow Paradigmcited by 31
  30. 2019X-SQL: reinforce schema representation with contextcited by 22
  31. 2024MAG-SQL: Multi-Agent Generative Approach with Soft Schema Linking and Iterative Sub-SQL Refinement for Text-to-SQLcited by 22
  32. 2023Text-to-SQL Error Correction with Language Models of Codecited by 21
  33. 2018Semantic Parsing with Syntax- and Table-Aware SQL Generationcited by 17
  34. 2024Improving Retrieval-augmented Text-to-SQL with AST-based Ranking and Schema Pruningcited by 16
  35. 2024Are LLMs Correctly Integrated into Software Systems?cited by 13
  36. 2024Insights into Natural Language Database Query Errors: From Attention Misalignment to User Handling Strategiescited by 11
  37. 2024SQLFixAgent: Towards Semantic-Accurate Text-to-SQL Parsing via Consistency-Enhanced Multi-Agent Collaborationcited by 10
  38. 2024ChatBI: Towards Natural Language to Complex Business Intelligence SQLcited by 9

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 9; 2026: 5.

14 papers.

Sort:

By area:

  1. 2025ReEx-SQL: Reasoning with Execution-Aware Reinforcement Learning for Text-to-SQLcited by 8
  2. 2025Query and Conquer: Execution-Guided SQL Generationcited by 5
  3. 2025SQL-of-Thought: Multi-agentic Text-to-SQL with Guided Error Correctioncited by 5
  4. 2026PV-SQL: Synergizing Database Probing and Rule-based Verification for Text-to-SQL Agentscited by 5
  5. 2025Chain-of-Query: Unleashing the Power of LLMs in SQL-Aided Table Understanding via Multi-Agent Collaborationcited by 4
  6. 2026Semantic Layers for Reliable LLM-Powered Data Analytics: A Paired Benchmark of Accuracy and Hallucination Across Three Frontier Modelscited by 3
  7. 2025SDE-SQL: Enhancing Text-to-SQL Generation in Large Language Models via Self-Driven Exploration with SQL Probescited by 2
  8. 2025Node-Level Uncertainty Estimation in LLM-Generated SQLcited by 2
  9. 2025GBV-SQL: Guided Generation and SQL2Text Back-Translation Validation for Multi-Agent Text2SQLcited by 1
  10. 2025SteinerSQL: Graph-Guided Mathematical Reasoning for Text-to-SQL Generationcited by 1
  11. 2026Both Ends Count! Just How Good are LLM Agents at "Text-to-Big SQL"?cited by 1
  12. 2025Leveraging RAG-LLMs for Urban Mobility Simulation and Analysiscited by 0
2 more
  1. 2026SemanticAgent: A Semantics-Aware Framework for Text-to-SQL Data Synthesiscited by 0
  2. 2026Causal Episodic Memory for Feedback-Driven Agent Repaircited by 0