mlsanityData to 14 Oct 2026

Trace Length is a Simple Uncertainty Signal in Reasoning Models

arXiv 2510.10409 ↗First posted 12 Oct 2025cs.AIcited by 19

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Uncertainty quantification for LLMs is a key research direction towards addressing hallucination and other issues that limit their reliable deployment. In this work, we show that reasoning trace length is a simple and useful confidence estimator in large reasoning models. Through comprehensive experiments across multiple models, datasets, and prompts, we show that trace length performs in comparable but complementary ways to other zero-shot confidence estimators such as verbalized confidence. Our work reveals that reasoning post-training fundamentally alters the relationship between trace length and accuracy, going beyond prior work that had shown that post-training causes traces to grow longer in general (e.g., "overthinking"). We investigate the mechanisms behind trace length's performance as a confidence signal, observing that the effect remains even after adjusting for confounders such as problem difficulty and GRPO-induced length bias. We identify high-entropy or "forking" tokens as playing a key role in the mechanism. Our findings demonstrate that reasoning post-training enhances uncertainty quantification beyond verbal expressions, and establish trace length as a practical confidence measure for large reasoning models.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 55 of its 84 references that are in the map. First posted on arXiv 12 Oct 2025.

Builds on

55 of 84 references.

Closest ancestors · 17

Same direction, newest first

  1. 2025Deep Think with Confidence21 Aug
  2. 2025Lexical Hints of Accuracy in LLM Reasoning Chains19 Aug
  3. 2025Towards Concise and Adaptive Thinking in Large Reasoning Models: A Survey13 Jul
  4. 2025Reasoning about Uncertainty: Do Reasoning Models Know When They Don't Know?22 Jun
  5. 2025ReCUT: Balancing Reasoning Length and Accuracy in LLMs via Stepwise Trails and Preference Optimization12 Jun
  6. 2025Wait, We Don't Need to "Wait"! Removing Thinking Tokens Improves Reasoning Efficiency10 Jun
  7. 2025Demystifying Reasoning Dynamics with Mutual Information: Thinking Tokens are Information Peaks in LLM Reasoning3 Jun
  8. 2025Don't Overthink it. Preferring Shorter Thinking Chains for Improved LLM Reasoning23 May
  9. 2025Reasoning Models Better Express Their Confidence20 May
  10. 2025ShorterBetter: Guiding Reasoning Models to Find Optimal Inference Length for Efficient Reasoning30 Apr
  11. 2025Thinking Out Loud: Do Reasoning Models Know When They're Right?9 Apr
  12. 2025Reasoning Models Know When They're Right: Probing Hidden States for Self-Verification7 Apr
5 more
  1. 2025Inference-Time Scaling for Complex Tasks: Where We Stand and What Lies Ahead31 Mar
  2. 2025Long Is More Important Than Difficult for Training Reasoning Models23 Mar
  3. 2025L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learning6 Mar
  4. 2025The relationship between reasoning and performance in large language models--o3 (mini) thinks harder, not longer21 Feb
  5. 2025Confidence Improves Self-Consistency in LLMs10 Feb

Other references · 38

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  3. 2024Qwen2.5 Technical Reportcited by 4,942
  4. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  5. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,356
  6. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,345
  7. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 810
  8. 2017Crowdsourcing Multiple Choice Science Questionscited by 765
  9. 2022Teaching Models to Express Their Uncertainty in Wordscited by 751
  10. 2025Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoningcited by 559
  11. 2025GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learningcited by 542
  12. 2025OpenThoughts: Data Recipes for Reasoning Modelscited by 304
26 more
  1. 2025DeepMath-103K: A Large-Scale, Challenging, Decontaminated, and Verifiable Mathematical Dataset for Advancing Reasoningcited by 280
  2. 2024Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMscited by 259
  3. 2025Why Language Models Hallucinatecited by 220
  4. 2025Skywork Open Reasoner 1 Technical Reportcited by 165
  5. 2024Large Language Models are Inconsistent and Biased Evaluatorscited by 153
  6. 2025AbstentionBench: Reasoning LLMs Fail on Unanswerable Questionscited by 145
  7. 2025Optimizing Test-Time Compute via Meta Reinforcement Fine-Tuningcited by 123
  8. 2025Beyond Binary Rewards: Training LMs to Reason About Their Uncertaintycited by 117
  9. 2025DeepSeek-R1 Thoughtology: Let's think about LLM Reasoningcited by 92
  10. 2025Pass@K Policy Optimization: Solving Harder Reinforcement Learning Problemscited by 87
  11. 2025Harnessing Multiple Large Language Models: A Survey on LLM Ensemblecited by 86
  12. 2025SuperGPQA: Scaling LLM Evaluation across 285 Graduate Disciplinescited by 79
  13. 2025Sample More to Think Less: Group Filtered Policy Optimization for Concise Reasoningcited by 75
  14. 2025ConfTuner: Training Large Language Models to Express Their Confidence Verballycited by 39
  15. 2025Supervised Fine Tuning on Curated Data is Reinforcement Learning (and can be improved)cited by 38
  16. 2025Position: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Traces!cited by 37
  17. 2025INTELLECT-2: A Reasoning Model Trained Through Globally Decentralized Reinforcement Learningcited by 25
  18. 2025Revisiting Uncertainty Estimation and Calibration of Large Language Modelscited by 23
  19. 2025The Illusion of Progress: Re-evaluating Hallucination Detection in LLMscited by 22
  20. 2025Uncalibrated Reasoning: GRPO Induces Overconfidence for Stochastic Outcomescited by 22
  21. 2025From Calibration to Collaboration: LLM Uncertainty Quantification Should Be More Human-Centeredcited by 21
  22. 2025LoVeC: Reinforcement Learning for Better Verbalized Confidence in Long-Form Generationscited by 17
  23. 2025Revisiting Epistemic Markers in Confidence Estimation: Can Markers Accurately Reflect Large Language Models' Uncertainty?cited by 16
  24. 2025Concise Reasoning, Big Gains: Pruning Long Reasoning Trace with Difficulty-Aware Promptingcited by 15
  25. 2024Reassessing How to Compare and Improve the Calibration of Machine Learning Modelscited by 8
  26. 2025Adjust for Trust: Mitigating Trust-Induced Inappropriate Reliance on AI Assistancecited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 17.

19 papers.

Sort:

By area:

  1. 2026How do LLMs Compute Verbal Confidencecited by 26
  2. 2026Are LLM Decisions Faithful to Verbal Confidence?cited by 8
  3. 2025DTS: Enhancing Large Reasoning Models via Decoding Tree Sketchingcited by 6
  4. 2026The Cost of Reasoning: Chain-of-Thought Induces Overconfidence in Vision-Language Modelscited by 5
  5. 2026Your Language Model is Its Own Critic: Reinforcement Learning with Value Estimation from Actor's Internal Statescited by 5
  6. 2025Let the Model Distribute Its Doubt: Confidence Estimation through Verbalized Probability Distributioncited by 3
  7. 2026SELFDOUBT: Uncertainty Quantification for Reasoning LLMs via the Hedge-to-Verify Ratiocited by 3
  8. 2026Recursive Language Models Meet Uncertainty: The Surprising Effectiveness of Self-Reflective Program Search for Long Contextcited by 2
  9. 2026On the effectiveness of reward functions in reinforcement learning for confidence calibration of large language modelscited by 2
  10. 2026Playing Psychic: Using Thought Trees to Predict Reasoning Models Accuracy on Coding Taskscited by 1
  11. 2026How Language Models Fail: Token-Level Signatures of Committed and Persistent Reasoning Failurescited by 1
  12. 2026Answer-Distribution Trajectories: A Stochastic-Dynamics View of LLM Reasoningcited by 1
7 more
  1. 2026Filtered Reasoning Score: Evaluating Reasoning Quality on a Model's Most-Confident Tracescited by 0
  2. 2026VERDI: Single-Call Confidence Estimation for Verification-Based LLM Judges via Decomposed Inferencecited by 0
  3. 2026Calibration, Not Answer Selection: Distilling Internal Confidence in Reasoning Modelscited by 0
  4. 2026UBTree: Parallel Tree Drafting via Unigram and Bigram Models for Speculative Decodingcited by 0
  5. 2026U-Space: Uncovering When and Why Uncertainty Arises in Language Modelscited by 0
  6. 2026Frustratingly Easy Majority Voting Using Length Conditioningcited by 0
  7. 2026Argument Graph Uncertainty: Quantifying Uncertainty from the Logical Structure of Reasoning Chainscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Quantifying Faithful Confidence Expression in Large Reasoning Models
  2. 2026Confidence Geometry Reveals Trace-Level Correctness in Large Language Model Reasoning
  3. 2026How Uncertainty Estimation Scales with Sampling in Reasoning Models
  4. 2026Calibration Drift Under Reasoning: How Chain-of-Thought Budgets Induce Overconfidence in Large Language Models
  5. 2025Think Just Enough: Sequence-Level Entropy as a Confidence Signal for LLM Reasoning
  6. 2026Quantized Reasoning Models Think They Need to Think Longer, but They Do Not
  7. 2026Probability is Not Enough: Exploring and Counting Divergent Tokens for Reasoning Uncertainty Quantification in LLMs
  8. 2026Understanding and Mitigating Premature Confidence for Better LLM Reasoning