mlsanityData to 7 Oct 2026

BaseCal: Unsupervised Confidence Calibration via Base Model Signals

arXiv 2601.03042 ↗First posted 6 Jan 2026cs.CLcited by 7

Not on the map. Why?

Abstract (arXiv)

Reliable confidence is essential for trusting the outputs of LLMs, yet widely deployed post-trained LLMs (PoLLMs) typically compromise this trust with severe overconfidence. In contrast, we observe that their corresponding base LLMs often remain well-calibrated. This naturally motivates us to calibrate PoLLM confidence using the base LLM as a reference. This work proposes two ways to achieve this. A straightforward solution, BaseCal-ReEval, evaluates PoLLM's responses by feeding them into the base LLM to get average probabilities as confidence. While effective, this approach introduces additional inference overhead. To address this, we propose BaseCal-Proj, which trains a lightweight projection to map the final-layer hidden states of PoLLMs back to those of their base LLMs. These projected states are then processed by the base LLM's output layer to derive base-calibrated confidence for PoLLM's responses. Notably, BaseCal is an unsupervised, plug-and-play solution that operates without human labels or LLM modifications. Experiments across five datasets and three LLM families demonstrate the effectiveness of BaseCal, reducing Expected Calibration Error (ECE) by an average of 42.90% compared to the best unsupervised baselines.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 31 of its 56 references that are in the map. First posted on arXiv 6 Jan 2026.

Builds on

31 of 56 references.

References · 31

  1. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  2. 2017On Calibration of Modern Neural Networkscited by 4,860
  3. 2024Qwen2.5 Technical Reportcited by 4,713
  4. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  5. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  6. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 946
  7. 2017Selective Classification for Deep Neural Networkscited by 799
  8. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 788
  9. 2022Teaching Models to Express Their Uncertainty in Wordscited by 731
  10. 2019Latent Retrieval for Weakly Supervised Open Domain Question Answeringcited by 691
  11. 2024Measuring short-form factuality in large language modelscited by 345
  12. 2023A Survey of Confidence Estimation and Calibration in Large Language Modelscited by 253
19 more
  1. 2024LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinationscited by 230
  2. 2024Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similaritiescited by 156
  3. 2024Taming Overconfidence in LLMs: Reward Calibration in RLHFcited by 105
  4. 2023On the Calibration of Large Language Models and Alignmentcited by 72
  5. 2024Factual Confidence of LLMs: on Reliability and Robustness of Current Estimatorscited by 59
  6. 2022A Close Look into the Calibration of Pre-trained Language Modelscited by 57
  7. 2024Calibrating Language Models with Adaptive Temperature Scalingcited by 57
  8. 2022Sample-dependent Adaptive Temperature Scaling for Improved Calibrationcited by 48
  9. 2024Calibrating the Confidence of Large Language Models by Eliciting Fidelitycited by 47
  10. 2024When to Trust LLMs: Aligning Confidence with Response Qualitycited by 42
  11. 2022Robust Calibration with Multi-domain Temperature Scalingcited by 39
  12. 2024Thermometer: Towards Universal Calibration for Large Language Modelscited by 38
  13. 2023Preserving Pre-trained Features Helps Calibrate Fine-tuned Language Modelscited by 29
  14. 2025Trust Me, I'm Wrong: LLMs Hallucinate with Certainty Despite Knowing the Answercited by 26
  15. 2024On Calibration of LLM-based Guard Models for Reliable Content Moderationcited by 21
  16. 2025Too Consistent to Detect: A Study of Self-Consistent Errors in LLMscited by 18
  17. 2025Improving Consistency in Large Language Models through Chain of Guidancecited by 13
  18. 2025Influences on LLM Calibration: A Study of Response Agreement, Loss Functions, and Prompt Stylescited by 12
  19. 2025Towards Objective Fine-tuning: How LLMs' Prior Knowledge Causes Potential Poor Calibration?cited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 7.

7 papers.

  1. 2026Unsupervised Confidence Calibration for Reasoning LLMs from a Single Generationcited by 2
  2. 2026Logical Consistency as a Bridge: Improving LLM Hallucination Detection via Label Constraint Modeling between Responses and Self-Judgmentscited by 0
  3. 2026VERDI: Single-Call Confidence Estimation for Verification-Based LLM Judges via Decomposed Inferencecited by 0
  4. 2026From token probabilities to calibrated confidence: An empirical study of mathematical question answeringcited by 0
  5. 2026SupportCal: Label-Free Calibration of Post-Trained LLMs via Reference Support and Corroborationcited by 0
  6. 2026Shared Experience, Separate Learning: Companion Confidence Calibration for LLMscited by 0
  7. 2026Guard Models Are Overconfident Where Base Models Are Uncertaincited by 0