mlsanityData to 13 Oct 2026

Text-to-SQL Calibration: No Need to Ask -- Just Rescale Model Probabilities

arXiv 2411.16742 ↗First posted 23 Nov 2024cs.DB cs.AI cs.IRcited by 7

Not on the map. Why?

Abstract (arXiv)

Calibration is crucial as large language models (LLMs) are increasingly deployed to convert natural language queries into SQL for commercial databases. In this work, we investigate calibration techniques for assigning confidence to generated SQL queries. We show that a straightforward baseline -- deriving confidence from the model's full-sequence probability -- outperforms recent methods that rely on follow-up prompts for self-checking and confidence verbalization. Our comprehensive evaluation, conducted across two widely-used Text-to-SQL benchmarks and multiple LLM architectures, provides valuable insights into the effectiveness of various calibration strategies.

Key facts. Cited by 7 papers in the map (data through 13 Oct 2026). Builds on 21 of its 29 references that are in the map. First posted on arXiv 23 Nov 2024.

Builds on

21 of 29 references.

References · 21

  1. 2017On Calibration of Modern Neural Networkscited by 4,902
  2. 2023Code Llama: Open Foundation Models for Codecited by 2,130
  3. 2022Language Models (Mostly) Know What They Knowcited by 1,823
  4. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,133
  5. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 960
  6. 2017Selective Classification for Deep Neural Networkscited by 825
  7. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 800
  8. 2023Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLscited by 680
  9. 2020Calibration of Pre-trained Transformerscited by 308
  10. 2024CodeS: Towards Building Open-source Language Models for Text-to-SQLcited by 186
  11. 2023MAC-SQL: A Multi-Agent Collaborative Framework for Text-to-SQLcited by 162
  12. 2023The Generative AI Paradox: "What It Can Create, It May Not Understand"cited by 98
9 more
  1. 2023Navigating the Grey Area: How Expressions of Uncertainty and Overconfidence Affect Language Modelscited by 91
  2. 2019Calibration of Encoder Decoder Models for Neural Machine Translationcited by 68
  3. 2023Uncertainty in Natural Language Generation: From Theory to Applicationscited by 61
  4. 2024Calibrating Language Models with Adaptive Temperature Scalingcited by 58
  5. 2022Calibrated Interpretation: Confidence Estimation in Semantic Parsingcited by 33
  6. 2023Benchmarking and Improving Text-to-SQL Generation under Ambiguitycited by 32
  7. 2023Zero and Few-shot Semantic Parsing with Ambiguous Inputscited by 16
  8. 2024I Need Help! Evaluating LLM's Ability to Ask for Users' Support: A Case Study on Text-to-SQL Generationcited by 11
  9. 2023TCE: A Test-Based Approach to Measuring Calibration Errorcited by 5

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2025: 4; 2026: 3.

7 papers.

  1. 2025ReFoRCE: A Text-to-SQL Agent with Self-Refinement, Consensus Enforcement, and Column Explorationcited by 36
  2. 2025ExCoT: Optimizing Reasoning for Text-to-SQL with Execution Feedbackcited by 24
  3. 2025Confidence Estimation for Text-to-SQL in Large Language Modelscited by 2
  4. 2025PaVeRL-SQL: Text-to-SQL via Partial-Match Rewards and Verbal Reinforcement Learningcited by 2
  5. 2026Calibrated Confidence Estimation for Tabular Question Answeringcited by 0
  6. 2026TrustDABench: Benchmarking Reliability and Robustness of LLMs for Structured Data Analysiscited by 0
  7. 2026Certified Against Which Oracle? Execution Labels Set the Reported Risk of Conformal Abstention for Text-to-SQLcited by 0