mlsanityData to 7 Oct 2026

From Natural Language to SQL: Review of LLM-based Text-to-SQL Systems

arXiv 2410.01066 ↗First posted 1 Oct 2024cs.CL cs.AIcited by 15

On the map: Retrieval-augmented generation → RAG pipeline design and evaluation

Abstract (arXiv)

LLMs when used with Retrieval Augmented Generation (RAG), are greatly improving the SOTA of translating natural language queries to structured and correct SQL. Unlike previous reviews, this survey provides a comprehensive study of the evolution of LLM-based text-to-SQL systems, from early rule-based models to advanced LLM approaches that use (RAG) systems. We discuss benchmarks, evaluation methods, and evaluation metrics. Also, we uniquely study the use of Graph RAGs for better contextual accuracy and schema linking in these systems. Finally, we highlight key challenges such as computational efficiency, model robustness, and data privacy toward improvements of LLM-based text-to-SQL systems.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Placed in RAG pipeline design and evaluation, one of 15 directions in Retrieval-augmented generation. Builds on 45 of its 61 references that are in the map. First posted on arXiv 1 Oct 2024.

Builds on

45 of 61 references.

References · 45

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  6. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  7. 2023A Survey of Large Language Modelscited by 2,537
  8. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,120
  9. 2017Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learningcited by 679
  10. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 667
  11. 2023DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correctioncited by 465
  12. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 457
33 more
  1. 2023Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluationcited by 378
  2. 2023RESDSQL: Decoupling Schema Linking and Skeleton Parsing for Text-to-SQLcited by 191
  3. 2023Large Language Models for Generative Information Extraction: A Surveycited by 183
  4. 2024CodeS: Towards Building Open-source Language Models for Text-to-SQLcited by 183
  5. 2019CoSQL: A Conversational Text-to-SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databasescited by 180
  6. 2023C3: Zero-shot Text-to-SQL with ChatGPTcited by 159
  7. 2019SParC: Cross-Domain Semantic Parsing in Contextcited by 141
  8. 2024Next-Generation Database Interfaces: A Survey of LLM-based Text-to-SQLcited by 118
  9. 2021KaggleDBQA: Realistic Evaluation of Text-to-SQL Parserscited by 105
  10. 2022Evaluating the Text-to-SQL Capabilities of Large Language Modelscited by 101
  11. 2021Exploring Underexplored Limitations of Cross-Domain Text-to-SQL Generalizationcited by 92
  12. 2024CLLMs: Consistency Large Language Modelscited by 71
  13. 2023ACT-SQL: In-Context Learning for Text-to-SQL with Automatically-Generated Chain-of-Thoughtcited by 65
  14. 2020On the Potential of Lexico-logical Alignments for Semantic Parsing to SQL Queriescited by 59
  15. 2024A Survey of Text-to-SQL in the Era of LLMs: Where are we, and where are we going?cited by 54
  16. 2024DTS-SQL: Decomposed Text-to-SQL with Small Large Language Modelscited by 45
  17. 2023ChatKBQA: A Generate-then-Retrieve Framework for Knowledge Base Question Answering with Fine-tuned Large Language Modelscited by 44
  18. 2024Middleware for LLMs: Tools Are Instrumental for Language Agents in Complex Environmentscited by 39
  19. 2023Symbol-LLM: Towards Foundational Symbol-centric Interface For Large Language Modelscited by 38
  20. 2024FinSQL: Model-Agnostic LLMs-based Text-to-SQL Framework for Financial Analysiscited by 35
  21. 2024Knowledge-to-SQL: Enhancing SQL Generation with Data Expert LLMcited by 35
  22. 2024SQLBench: A Comprehensive Evaluation for Text-to-SQL Capabilities of Large Language Modelscited by 32
  23. 2022Towards Robustness of Text-to-SQL Models Against Natural and Realistic Adversarial Table Perturbationcited by 28
  24. 2023A Benchmark to Understand the Role of Knowledge Graphs on Large Language Model's Accuracy for Question Answering on Enterprise SQL Databasescited by 25
  25. 2024Interactive-KBQA: Multi-Turn Interactions for Knowledge Base Question Answering with Large Language Modelscited by 25
  26. 2024Metasql: A Generate-then-Rank Framework for Natural Language to SQL Translationcited by 17
  27. 2019Content Enhanced BERT-based Text-to-SQL Generationcited by 16
  28. 2024SQL-GEN: Bridging the Dialect Gap for Text-to-SQL Via Synthetic Data And Model Mergingcited by 16
  29. 2022Deep Learning Driven Natural Languages Text to SQL Query Conversion: A Surveycited by 11
  30. 2024Benchmarking the Performance of Pre-trained LLMs across Urdu NLP Taskscited by 8
  31. 2024SQLfuse: Enhancing Text-to-SQL Performance through Comprehensive LLM Synergycited by 7
  32. 2024Use of a Structured Knowledge Base Enhances Metadata Curation by Large Language Modelscited by 2
  33. 2024Late Breaking Results: Fortifying Neural Networks: Safeguarding Against Adversarial Attacks with Stochastic Computingcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 8; 2026: 7.

15 papers.

Sort:

By area:

  1. 2025DeepAnalyze: Agentic Large Language Models for Autonomous Data Sciencecited by 49
  2. 2025RAG over Tables: Hierarchical Memory Index, Multi-Stage Retrieval, and Benchmarkingcited by 17
  3. 2026From Failure to Mastery: Generating Hard Samples for Tool-use Agentscited by 8
  4. 2025TaTToo: Tool-Grounded Thinking PRM for Test-Time Scaling in Tabular Reasoningcited by 5
  5. 2025SAFE-SQL: Self-Augmented In-Context Learning with Fine-grained Example Selection for Text-to-SQLcited by 4
  6. 2025LLM Agents Are the Antidote to Walled Gardenscited by 3
  7. 2026AgentNLQ: A General-Purpose Agent for Natural Language to SQLcited by 2
  8. 2025FinStat2SQL: A Text2SQL Pipeline for Financial Statement Analysiscited by 1
  9. 2025Agentic LLMs for Question Answering over Tabular Datacited by 1
  10. 2025From Natural Language to Control Signals: A Conceptual Framework for Semantic Channel Finding in Complex Experimental Infrastructurecited by 1
  11. 2026NL2SQLBench: A Modular Benchmarking Framework for LLM-Enabled NL2SQL Solutionscited by 1
  12. 2026CBR-to-SQL: Rethinking Retrieval-based Text-to-SQL using Case-based Reasoning in the Healthcare Domaincited by 0
3 more
  1. 2026Neurosymbolics for Data Engineering: Achieving Long Context Token Reduction Without Finetuningcited by 0
  2. 2026A Hybrid Agentic AI Framework for Intelligent Supply Chain Analyticscited by 0
  3. 2026Natural Language to First-Order Logic LLM-based Autoformalizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026CSR-RAG: An Efficient Retrieval System for Text-to-SQL on the Enterprise Scale
  2. 2025Balancing Content Size in RAG-Text2SQL System
  3. 2025Datrics Text2SQL: A Framework for Natural Language to SQL Query Generation
  4. 2024T-RAG: Lessons from the LLM Trenches
  5. 2024Lucy: Think and Reason to Solve Text-to-SQL
  6. 2026Integrating Reasoning and Generalization in Text-to-SQL via Self-Enhanced Fine-Tuning
  7. 2024RB-SQL: A Retrieval-based LLM Framework for Text-to-SQL
  8. 2025Weaver: Interweaving SQL and LLM for Table Reasoning