mlsanityData to 14 Oct 2026

Confidence Estimation for Text-to-SQL in Large Language Models

arXiv 2508.14056 ↗First posted 8 Aug 2025cs.CL cs.DBcited by 3

Not on the map. Why?

Abstract (arXiv)

Confidence estimation for text-to-SQL aims to assess the reliability of model-generated SQL queries without having access to gold answers. We study this problem in the context of large language models (LLMs), where access to model weights and gradients is often constrained. We explore both black-box and white-box confidence estimation strategies, evaluating their effectiveness on cross-domain text-to-SQL benchmarks. Our evaluation highlights the superior performance of consistency-based methods among black-box models and the advantage of SQL-syntax-aware approaches for interpreting LLM logits in white-box settings. Furthermore, we show that execution-based grounding of queries provides a valuable supplementary signal, improving the effectiveness of both approaches.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 31 of its 41 references that are in the map. First posted on arXiv 8 Aug 2025.

Builds on

31 of 41 references.

References · 31

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  5. 2017On Calibration of Modern Neural Networkscited by 4,960
  6. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  7. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  8. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  9. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  10. 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,391
  11. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,328
  12. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,181
19 more
  1. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,139
  2. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 974
  3. 2022Teaching Models to Express Their Uncertainty in Wordscited by 751
  4. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 688
  5. 2023DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correctioncited by 477
  6. 2019RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parserscited by 424
  7. 2023Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluationcited by 385
  8. 2019Towards Complex Text-to-SQL in Cross-Domain Database with Intermediate Representationcited by 210
  9. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 206
  10. 2023C3: Zero-shot Text-to-SQL with ChatGPTcited by 164
  11. 2021LGESQL: Line Graph Enhanced Text-to-SQL Model with Mixed Local and Non-Local Relationscited by 119
  12. 2023Llamas Know What GPTs Don't Show: Surrogate Models for Confidence Estimationcited by 39
  13. 2024Understanding the Effects of Noise in Text-to-SQL: An Examination of the BIRD-Bench Benchmarkcited by 34
  14. 2022Calibrated Interpretation: Confidence Estimation in Semantic Parsingcited by 33
  15. 2024Cycles of Thought: Measuring LLM Confidence through Stable Explanationscited by 29
  16. 2025Reliable Text-to-SQL with Adaptive Abstentioncited by 20
  17. 2020Did You Ask a Good Question? A Cross-Domain Question Intention Classification Benchmark for Text-to-SQLcited by 16
  18. 2025Confidence Estimation for Error Detection in Text-to-SQL Systemscited by 11
  19. 2024Text-to-SQL Calibration: No Need to Ask -- Just Rescale Model Probabilitiescited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025Towards Agents That Know When They Don't Know: Uncertainty as a Control Signal for Structured Reasoningcited by 10
  2. 2026Certified Against Which Oracle? Execution Labels Set the Reported Risk of Conformal Abstention for Text-to-SQLcited by 0
  3. 2026Does a Governed Confidence Score Know When a Text-to-SQL Answer Is Right? An Exploratory Study on BIRD and a Pre-registered Confirmation on Spidercited by 0