mlsanityData to 7 Oct 2026

Also Small Models Can Reasonably Self-Evaluate Their Confidence

arXiv 2609.39478 ↗First posted 30 Sep 2026cs.LGcited by 0

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

This study systematically evaluates self-evaluation-based uncertainty quantification across different language models of varying sizes on question-answering tasks spanning general to specialized knowledge domains. Using various self-evaluation methods where models judge their own predictions, we examine how model scale and domain specificity affect the quality of self-assessed confidence signals. Our results reveal that while accuracy predictably declines with smaller models and more specialized domains, the reliability of self-evaluated confidence remains largely stable across both dimensions. This independence means the most capable model is not necessarily the best at self-assessing prediction reliability. These findings suggest that smaller models can achieve reasonable self-assessed confidence despite lower accuracy, making them viable for resource-constrained deployments.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 14 of its 26 references that are in the map. First posted on arXiv 30 Sep 2026.

Builds on

14 of 26 references.

References · 14

  1. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  2. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  3. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  4. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,258
  5. 2023Chain-of-Verification Reduces Hallucination in Large Language Modelscited by 317
  6. 2026Gemma 4 Technical Reportcited by 187
  7. 2023LM-Polygraph: Uncertainty Estimation for Language Modelscited by 130
  8. 2023Self-contradictory Hallucinations of Large Language Models: Evaluation, Detection and Mitigationcited by 123
  9. 2026Ministral 3cited by 88
  10. 2023Towards Mitigating Hallucination in Large Language Models via Self-Reflectioncited by 81
  11. 2024LLM Internal States Reveal Hallucination Risk Faced With a Querycited by 80
  12. 2024Self-Alignment for Factuality: Mitigating Hallucinations in LLMs via Self-Evaluationcited by 79
2 more
  1. 2023Self-Evaluation Improves Selective Generation in Large Language Modelscited by 74
  2. 2020Uncertainty Estimation in Autoregressive Structured Predictioncited by 21

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Confidence Under the Hood: An Investigation into the Confidence-Probability Alignment in Large Language Models
  2. 2026An Analysis of Training-Free Self-Reported Confidence in Language Models
  3. 2024Confidence in the Reasoning of Large Language Models
  4. 2026LLMs Should Express Uncertainty Explicitly
  5. 2026Latent Confidence Alignment for LLM Self-Assessment
  6. 2026Rethinking Uncertainty Evaluation in Large Language Models
  7. 2026CALIBER: Calibrating Confidence Before and After Reasoning in Language Models
  8. 2026Confident in a Confidence Score: Investigating the Sensitivity of Confidence Scores to Supervised Fine-Tuning