# Not All Tokens Are What You Need In Thinking

arXiv: https://arxiv.org/abs/2505.17827 · HTML: https://mlsanity.com/paper/2505.17827/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 21 of its 32 references that are in the map.
- First posted on arXiv 23 May 2025.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)

## Builds on (most cited first, 21 of 21)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [Kimi k1.5: Scaling Reinforcement Learning with LLMs](https://mlsanity.com/paper/2501.12599/) (2025)
- [Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs](https://mlsanity.com/paper/2412.21187/) (2024)
- [Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models](https://mlsanity.com/paper/2503.16419/) (2025)
- [LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression](https://mlsanity.com/paper/2310.06839/) (2023)
- [LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compression](https://mlsanity.com/paper/2403.12968/) (2024)
- [Demystifying Long Chain-of-Thought Reasoning in LLMs](https://mlsanity.com/paper/2502.03373/) (2025)
- [Chain of Draft: Thinking Faster by Writing Less](https://mlsanity.com/paper/2502.18600/) (2025)
- [A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?](https://mlsanity.com/paper/2503.24235/) (2025)
- [Reasoning Models Can Be Effective Without Thinking](https://mlsanity.com/paper/2504.09858/) (2025)
- [AIMO-2 Winning Solution: Building State-of-the-Art Mathematical Reasoning Models with OpenMathReasoning dataset](https://mlsanity.com/paper/2504.16891/) (2025)
- [C3oT: Generating Shorter Chain-of-Thought without Compromising Effectiveness](https://mlsanity.com/paper/2412.11664/) (2024)
- [How Well do LLMs Compress Their Own Chain-of-Thought? A Token Complexity Approach](https://mlsanity.com/paper/2503.01141/) (2025)
- [Effectively Controlling Reasoning Models through Thinking Intervention](https://mlsanity.com/paper/2503.24370/) (2025)
- [1.4 Million Open-Source Distilled Reasoning Dataset to Empower Large Language Model Training](https://mlsanity.com/paper/2503.19633/) (2025)
- [Think Twice: Enhancing LLM Reasoning by Scaling Multi-round Test-time Thinking](https://mlsanity.com/paper/2503.19855/) (2025)

## Built on by (most cited first, 20 of 20)

- [Don't Overthink It: A Survey of Efficient R1-style Large Reasoning Models](https://mlsanity.com/paper/2508.02120/) (2025)
- [DRPO: Efficient Reasoning via Decoupled Reward Policy Optimization](https://mlsanity.com/paper/2510.04474/) (2025)
- [Replaying pre-training data improves fine-tuning](https://mlsanity.com/paper/2603.04964/) (2026)
- [Neural Chain-of-Thought Search: Searching the Optimal Reasoning Path to Enhance Large Language Models](https://mlsanity.com/paper/2601.11340/) (2026)
- [Towards Efficient Large Language Reasoning Models via Extreme-Ratio Chain-of-Thought Compression](https://mlsanity.com/paper/2602.08324/) (2026)
- [PEAR: Phase Entropy Aware Reward for Efficient Reasoning](https://mlsanity.com/paper/2510.08026/) (2025)
- [Steering Large Reasoning Models towards Concise Reasoning via Flow Matching](https://mlsanity.com/paper/2602.05539/) (2026)
- [Efficient Reasoning with Balanced Thinking](https://mlsanity.com/paper/2603.12372/) (2026)
- [ThinkPilot: Steering Reasoning Models via Automated Think-prefixes Optimization](https://mlsanity.com/paper/2510.12063/) (2025)
- [Draft-Thinking: Learning Efficient Reasoning in Long Chain-of-Thought LLMs](https://mlsanity.com/paper/2603.00578/) (2026)
- [Learning from Partial Chain-of-Thought via Truncated-Reasoning Self-Distillation](https://mlsanity.com/paper/2603.13274/) (2026)
- [Shorthand for Thought: Compressing LLM Reasoning via Entropy-Guided Supertokens](https://mlsanity.com/paper/2604.26355/) (2026)
- [Focused Chain-of-Thought: Efficient LLM Reasoning via Structured Input Information](https://mlsanity.com/paper/2511.22176/) (2025)
- [A State-Transition Framework for Efficient LLM Reasoning](https://mlsanity.com/paper/2602.01198/) (2026)
- [Reasoning Denoiser: Denoising Reasoning Traces for Hallucination Detection in Large Reasoning Models](https://mlsanity.com/paper/2607.22098/) (2026)
- [Bridging Efficiency and Transparency: Explainable CoT Compression in Multimodal Large Reasoning Models](https://mlsanity.com/paper/2602.09485/) (2026)
- [SkillRater: Untangling Capabilities in Multimodal Data](https://mlsanity.com/paper/2602.11615/) (2026)
- [Stateful Reasoning via Insight Replay](https://mlsanity.com/paper/2605.14457/) (2026)
- [Revisiting Complete Reasoning Traces for Post-Training](https://mlsanity.com/paper/2609.07103/) (2026)
- [Efficient Reasoning via Constrained Optimization in Latent Space](https://mlsanity.com/paper/2609.34181/) (2026)

## Abstract (arXiv)

Modern reasoning models, such as OpenAI's o1 and DeepSeek-R1, exhibit impressive problem-solving capabilities but suffer from critical inefficiencies: high inference latency, excessive computational resource consumption, and a tendency toward overthinking -- generating verbose chains of thought (CoT) laden with redundant tokens that contribute minimally to the final answer. To address these issues, we propose Conditional Token Selection (CTS), a token-level compression framework with a flexible and variable compression ratio that identifies and preserves only the most essential tokens in CoT. CTS evaluates each token's contribution to deriving correct answers using conditional importance scoring, then trains models on compressed CoT. Extensive experiments demonstrate that CTS effectively compresses long CoT while maintaining strong reasoning performance. Notably, on the GPQA benchmark, Qwen2.5-14B-Instruct trained with CTS achieves a 9.1% accuracy improvement with 13.2% fewer reasoning tokens (13% training token reduction). Further reducing training tokens by 42% incurs only a marginal 5% accuracy drop while yielding a 75.8% reduction in reasoning tokens, highlighting the prevalence of redundancy in existing CoT.