mlsanityData to 14 Oct 2026

Agentic-SQL Revisited: Autonomy-Based Taxonomy and Empirical Benchmark Analysis for LLM Text-to-SQL

arXiv 2608.15389 ↗First posted 15 Aug 2026cs.AIcited by 1

On the map: LLM agents and tool use → Web, search, and research agents

Abstract (arXiv)

LLM-based Text-to-SQL progress is reported across heterogeneous benchmarks, backbones, and inference protocols, making cross-system comparison fragile. We reframe the field as a leaderboard aggregation: we collect the metrics authors themselves report and organize them along an inference-autonomy axis spanning constrained, in-context, iterative, agentic, and reasoning-internalized generation, with traceable provenance for every cell. To anchor the aggregation empirically, we run a focused case study on Spider, comparing 8B open-source backbones with and without chain-of-thought (CoT) supervision against few-shot DeepSeek~V3 and GLM-4 baselines. Four patterns emerge: Spider gains transfer unevenly to BIRD and Spider~2.0; autonomy buys robustness at non-trivial cost; reasoning internalization sits between answer-only decoding and externally orchestrated agents; and CoT gains concentrate on Hard and Extra-Hard queries. We release a Python harness mirroring the autonomy axis so that future methods can be added directly to the leaderboard.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 29 of its 34 references that are in the map. First posted on arXiv 15 Aug 2026.

Builds on

29 of 34 references.

Closest ancestors · 2

Same direction, newest first

  1. 2026BAP-SQL: Budget-Aware Observation Planning for Agentic Text-to-SQL3 Aug
  2. 2023MAC-SQL: A Multi-Agent Collaborative Framework for Text-to-SQL18 Dec

Other references · 27

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  4. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  5. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  6. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,139
  7. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 688
  8. 2023DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correctioncited by 477
  9. 2019RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parserscited by 424
  10. 2021PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Modelscited by 411
  11. 2023Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluationcited by 385
  12. 2024Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflowscited by 232
15 more
  1. 2023RESDSQL: Decoupling Schema Linking and Skeleton Parsing for Text-to-SQLcited by 195
  2. 2024CHESS: Contextual Harnessing for Efficient SQL Synthesiscited by 169
  3. 2024Next-Generation Database Interfaces: A Survey of LLM-based Text-to-SQLcited by 120
  4. 2019A Comprehensive Exploration on WikiSQL with Table-Aware Word Contextualizationcited by 88
  5. 2024DTS-SQL: Decomposed Text-to-SQL with Small Large Language Modelscited by 45
  6. 2024Text2SQL is Not Enough: Unifying AI and Databases with TAGcited by 34
  7. 2025STaR-SQL: Self-Taught Reasoner for Text-to-SQLcited by 17
  8. 2026Does RAG Know When Retrieval Is Wrong? Diagnosing Context Compliance under Knowledge Conflictcited by 15
  9. 2026Relevant Is Not Warranted: Evidence-Force Calibration for Cited RAGcited by 14
  10. 2025EllieSQL: Cost-Efficient Text-to-SQL with Complexity-Aware Routingcited by 13
  11. 2025ExeSQL: Self-Taught Text-to-SQL Models with Execution-Driven Bootstrapping for SQL Dialectscited by 12
  12. 2026Phantom Guardrails: When Self-Improving Agent Harnesses Fix Failures That Never Happenedcited by 7
  13. 2024EHR-SeqSQL : A Sequential Text-to-SQL Dataset For Interactively Exploring Electronic Health Recordscited by 5
  14. 2026When Synthetic Users Fail: A Cross-Domain Benchmark of LLM-Simulated Human Survey Responsescited by 5
  15. 2023Stand for Something or Fall for Everything: Predict Misinformation Spread with Stance-Aware Graph Neural Networkscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Style Over Substance: Content-Invariant Wrappers Flip LLM Safety-Judge Verdictscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025BAPPA: Benchmarking Agents, Plans, and Pipelines for Automated Text-to-SQL Generation
  2. 2026AgentSM: Semantic Memory for Agentic Text-to-SQL
  3. 2026BADGER: Bridging Agentic and Deterministic Evaluation for Generative Enterprise Reasoning
  4. 2025AGENTIQL: An Agent-Inspired Multi-Expert Framework for Text-to-SQL Generation
  5. 2025RAISE: Reasoning Agent for Interactive SQL Exploration
  6. 2026RoboPhD: Self-Improving Text-to-SQL Through Autonomous Agent Evolution
  7. 2025Rethinking Agentic Workflows: Evaluating Inference-Based Test-Time Scaling Strategies in Text2SQL Tasks
  8. 2025From Queries to Insights: Agentic LLM Pipelines for Spatio-Temporal Text-to-SQL