mlsanityData to 7 Oct 2026

Reasoning Can Be Restored by Correcting a Few Decision Tokens

arXiv 2605.16874 ↗First posted 16 May 2026cs.AIcited by 2

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Large reasoning models (LRMs) substantially outperform their base LLM counterparts on challenging reasoning benchmarks, yet it remains poorly understood where base models go wrong during token-by-token generation and how to narrow this gap efficiently. We study the base-reasoning gap through quantifying token-level distributional disagreement between a base model and a stronger reasoning model using likelihood-based divergences. Across benchmarks, we find that the reasoning advantage is highly sparse and concentrates on a small set of early, planning-related decision tokens. For instance, on Qwen3-0.6B, only ~8% of generated tokens account for the salient disagreement, and these tokens concentrate early in the response, are strongly enriched in planning-related decisions (17x), and coincide with high base-model uncertainty -- suggesting that base models fail mainly at early planning points that steer the subsequent reasoning trajectory. Building on these findings, we propose disagreement-guided token intervention, a simple inference-time delegation scheme that performs a one-token takeover by the reasoning model only at high-disagreement positions and immediately switches back to the base model. With a small intervention budget, this sparse delegation substantially recovers and can even surpass the performance of a same-size reasoning model on challenging reasoning tasks. Code is available at https://github.com/AlphaLab-USTC/RRTokenIntervention.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 42 of its 55 references that are in the map. First posted on arXiv 16 May 2026.

Builds on

42 of 55 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Reasoning-Finetuning Repurposes Latent Representations in Base Models16 Jul
  2. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Models17 May
  3. 2023Self-Refine: Iterative Refinement with Self-Feedback30 Mar

Other references · 39

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2024The Llama 3 Herd of Modelscited by 14,192
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  6. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  7. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  8. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  9. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  10. 2025Qwen3 Technical Reportcited by 8,195
  11. 2020Scaling Laws for Neural Language Modelscited by 6,411
  12. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
27 more
  1. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  2. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  3. 2023Let's Verify Step by Stepcited by 3,598
  4. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,748
  5. 2023A Survey of Large Language Modelscited by 2,537
  6. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
  7. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  8. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  9. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,099
  10. 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 840
  11. 2023Steering Language Models With Activation Engineeringcited by 824
  12. 2024Self-Rewarding Language Modelscited by 654
  13. 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 633
  14. 2025Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoningcited by 540
  15. 2025Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRscited by 373
  16. 2022Large Language Models Are Reasoning Teacherscited by 318
  17. 2025Reinforcement Learning for Reasoning in Large Language Models with One Training Examplecited by 288
  18. 2025Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMscited by 276
  19. 2025Echo Chamber: RL Post-training Amplifies Behaviors Learned in Pretrainingcited by 123
  20. 2024Speculative Knowledge Distillation: Bridging the Teacher-Student Gap Through Interleaved Samplingcited by 85
  21. 2023Cascade Speculative Drafting for Even Faster LLM Inferencecited by 83
  22. 2025First Return, Entropy-Eliciting Explorecited by 45
  23. 2025Base Models Know How to Reason, Thinking Models Learn Whencited by 25
  24. 2025On Predictability of Reinforcement Learning Dynamics for Large Language Modelscited by 20
  25. 2025Reasoning-as-Logic-Units: Scaling Test-Time Reasoning in Large Language Models Through Logic Unit Alignmentcited by 12
  26. 2024AdaSwitch: Adaptive Switching between Small and Large Agents for Effective Cloud-Local Collaborative Learningcited by 5
  27. 2025Beyond Correctness: Confidence-Aware Reward Modeling for Enhancing Large Language Model Reasoningcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Finding the Evidence: Discovering Decision-Supporting Tokens for On-Policy Reasoning Distillationcited by 5
  2. 2026Shattering the Autoregressive Curse: Dynamic Epistemic Entropy Orchestrated Erasable Reinforcement Learning for LLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026The Reasoning Tax: Token Economics of LLM Reasoning Across Task Types and Deployment Contexts
  2. 2025Do Thinking Tokens Help or Trap? Towards More Efficient Large Reasoning Model
  3. 2026First Token Matters: Understanding Safety Collapse in Large Reasoning Models
  4. 2026On the Token Value Inequality in Efficient Reasoning
  5. 2025On Reasoning Strength Planning in Large Reasoning Models
  6. 2026Improving reasoning at inference time via uncertainty minimisation
  7. 2025Not All Tokens Matter: Towards Efficient LLM Reasoning via Token Significance in Reinforcement Learning
  8. 2026RFEval: Benchmarking Reasoning Faithfulness under Counterfactual Reasoning Intervention in Large Reasoning Models