mlsanityData to 14 Oct 2026

Can Confidence Estimates Decide When Chain-of-Thought Is Necessary for LLMs?

arXiv 2510.21007 ↗First posted 23 Oct 2025cs.CLcited by 2

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

Abstract (arXiv)

Chain-of-thought (CoT) prompting is a common technique for improving the reasoning abilities of large language models (LLMs). However, extended reasoning is often unnecessary and substantially increases token usage. As such, a key question becomes how to optimally allocate compute to when reasoning is actually needed. We study this through confidence-gated CoT, where a model produces a direct answer and a confidence estimate to decide whether to invoke CoT. We present an evaluation framework together with the first systematic study of confidence signals for this decision. We evaluate four representative confidence measures and compare them with random gating and an oracle upper bound. Experiments across two model families and diverse reasoning tasks show that existing training-free confidence measures can reduce redundant reasoning. However, we also find that the utility of individual confidence measures is inconsistent across settings. Through our evaluation framework and analysis, our study provides practical guidance toward developing and evaluating models that selectively use CoT.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 43 of its 54 references that are in the map. First posted on arXiv 23 Oct 2025.

Builds on

43 of 54 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025Token Signature: Predicting Chain-of-Thought Gains with Token Decoding Feature in Large Language Models6 Jun
  2. 2025Ada-R1: Hybrid-CoT via Bi-Level Adaptive Reasoning Optimization30 Apr
  3. 2024Mind Your Step (by Step): Chain-of-Thought can Reduce Performance on Tasks where Thinking Makes Humans Worse27 Oct
  4. 2024To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning18 Sep
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Models28 Jan

Other references · 38

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  3. 2017On Calibration of Modern Neural Networkscited by 4,960
  4. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  5. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  6. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,822
  7. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,386
  8. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 974
  9. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 819
  10. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 812
  11. 2021Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategiescited by 743
  12. 2023AGIEval: A Human-Centric Benchmark for Evaluating Foundation Modelscited by 716
26 more
  1. 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 682
  2. 2023Generating with Confidence: Uncertainty Quantification for Black-box Large Language Modelscited by 293
  3. 2024Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMscited by 259
  4. 2025Dynamic Early Exit in Reasoning Modelscited by 219
  5. 2023MuSR: Testing the Limits of Chain-of-thought with Multistep Soft Reasoningcited by 201
  6. 2024Are We Done with MMLU?cited by 194
  7. 2024GraphRouter: A Graph-based Router for LLM Selectionscited by 165
  8. 2025Reasoning Models Know When They're Right: Probing Hidden States for Self-Verificationcited by 159
  9. 2025DAST: Difficulty-Adaptive Slow-Thinking for Large Reasoning Modelscited by 158
  10. 2023Shifting Attention to Relevance: Towards the Predictive Uncertainty Quantification of Free-Form Large Language Modelscited by 150
  11. 2025Beyond Binary Rewards: Training LMs to Reason About Their Uncertaintycited by 117
  12. 2024On Verbalized Confidence Scores for LLMscited by 83
  13. 2025Learn to Reason Efficiently with Adaptive Length-based Reward Shapingcited by 75
  14. 2024Learning to Route LLMs with Confidence Tokenscited by 67
  15. 2025Think Only When You Need with Large Hybrid-Reasoning Modelscited by 66
  16. 2025Don't Overthink It: A Survey of Efficient R1-style Large Reasoning Modelscited by 37
  17. 2025Prolonged Reasoning Is Not All You Need: Certainty-Based Adaptive Routing for Efficient LLM/MLLM Reasoningcited by 35
  18. 2024Optimising Calls to Large Language Models with Uncertainty-Based Two-Tier Selectioncited by 33
  19. 2025Mitigating Overthinking in Large Reasoning Models via Manifold Steeringcited by 33
  20. 2025Uncertainty-Guided Chain-of-Thought for Code Generation with LLMscited by 32
  21. 2025Confident or Seek Stronger: Exploring Uncertainty-Based On-device LLM Routing From Benchmarking to Generalizationcited by 29
  22. 2025Incentivizing Dual Process Thinking for Efficient Large Language Model Reasoningcited by 21
  23. 2025Understanding Model Calibration -- A gentle introduction and visual exploration of calibration and the expected calibration error (ECE)cited by 20
  24. 2025HAPO: Training Language Models to Reason Concisely via History-Aware Policy Optimizationcited by 19
  25. 2025LoVeC: Reinforcement Learning for Better Verbalized Confidence in Long-Form Generationscited by 17
  26. 2025Adaptive Deep Reasoning: Triggering Deep Thinking When Neededcited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025Colon-X: Advancing Intelligent Colonoscopy toward Clinical Reasoningcited by 1
  2. 2026When Should Agents Think? Adaptive Reasoning via Cross-Turn Estimationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens
  2. 2024Chain-of-Thought Reasoning Without Prompting
  3. 2023Stress Testing Chain-of-Thought Prompting for Large Language Models
  4. 2024The Impact of Reasoning Step Length on Large Language Models
  5. 2022Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters
  6. 2025Unveiling Confirmation Bias in Chain-of-Thought Reasoning
  7. 2025Causal Sufficiency and Necessity Improves Chain-of-Thought Reasoning
  8. 2024Deciphering the Factors Influencing the Efficacy of Chain-of-Thought: Probability, Memorization, and Noisy Reasoning