mlsanityData to 7 Oct 2026

TokenSkip: Controllable Chain-of-Thought Compression in LLMs

arXiv 2502.12067 ↗First posted 17 Feb 2025cs.CL cs.AIcited by 246

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Chain-of-Thought (CoT) has been proven effective in enhancing the reasoning capabilities of large language models (LLMs). Recent advancements, such as OpenAI's o1 and DeepSeek-R1, suggest that scaling up the length of CoT sequences during inference could further boost LLM reasoning performance. However, due to the autoregressive nature of LLM decoding, longer CoT outputs lead to a linear increase in inference latency, adversely affecting user experience, particularly when the CoT exceeds 10,000 tokens. To address this limitation, we analyze the semantic importance of tokens within CoT outputs and reveal that their contributions to reasoning vary. Building on this insight, we propose TokenSkip, a simple yet effective approach that enables LLMs to selectively skip less important tokens, allowing for controllable CoT compression. Extensive experiments across various models and tasks demonstrate the effectiveness of TokenSkip in reducing CoT token usage while preserving strong reasoning performance. Notably, when applied to Qwen2.5-14B-Instruct, TokenSkip reduces reasoning tokens by 40% (from 313 to 181) on GSM8K, with less than a 0.4% performance drop. We release our code and checkpoints in https://github.com/hemingkx/TokenSkip.

Key facts. Cited by 246 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 24 of its 43 references that are in the map. First posted on arXiv 17 Feb 2025.

Builds on

24 of 43 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Compressed Chain of Thought: Efficient Reasoning Through Dense Representations17 Dec
  2. 2024C3oT: Generating Shorter Chain-of-Thought without Compromising Effectiveness16 Dec
  3. 2023Think before you speak: Training Language Models With Pause Tokens3 Oct

Other references · 21

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  4. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  5. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  6. 2023Let's Verify Step by Stepcited by 3,598
  7. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,793
  8. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,442
  9. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 871
  10. 2023Learning to Compress Prompts with Gist Tokenscited by 379
  11. 2023Adapting Language Models to Compress Contextscited by 350
  12. 2024LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compressioncited by 343
9 more
  1. 2024From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Stepcited by 225
  2. 2024The Impact of Reasoning Step Length on Large Language Modelscited by 152
  3. 2022Prompt Compression and Contrastive Conditioning for Controllability and Toxicity Reduction in Language Modelscited by 96
  4. 2023Not All Neuro-Symbolic Concepts Are Created Equal: Analysis and Mitigation of Reasoning Shortcutscited by 63
  5. 2024Can Language Models Learn to Skip Steps?cited by 59
  6. 2024Learning to Compress Prompt in Natural Language Formatscited by 54
  7. 2022Token Dropping for Efficient BERT Pretrainingcited by 42
  8. 2024Break the Chain: Large Language Models Can be Shortcut Reasonerscited by 32
  9. 2024What Are Step-Level Reward Models Rewarding? Counterintuitive Findings from MCTS-Boosted Mathematical Reasoningcited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 146; 2026: 100.

246 papers.

Sort:

By area:

  1. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Modelscited by 456
  2. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  3. 2025Dynamic Early Exit in Reasoning Modelscited by 217
  4. 2025Reasoning Models Can Be Effective Without Thinkingcited by 182
  5. 2025LightMem: Lightweight and Efficient Memory-Augmented Generationcited by 163
  6. 2025DAST: Difficulty-Adaptive Slow-Thinking for Large Reasoning Modelscited by 154
  7. 2025LLM Post-Training: A Deep Dive into Reasoning Large Language Modelscited by 129
  8. 2025CPPO: Accelerating the Training of Group Relative Policy Optimization-Based Reasoning Modelscited by 93
  9. 2025Efficient Reasoning Models: A Surveycited by 93
  10. 2025S-GRPO: Early Exit via Reinforcement Learning in Reasoning Modelscited by 87
  11. 2025AdaCoT: Pareto-Optimal Adaptive Chain-of-Thought Triggering via Reinforcement Learningcited by 86
  12. 2025Think Silently, Think Fast: Dynamic Latent Compression of LLM Reasoning Chainscited by 81
38 more
  1. 2025Learn to Reason Efficiently with Adaptive Length-based Reward Shapingcited by 74
  2. 2025ShorterBetter: Guiding Reasoning Models to Find Optimal Inference Length for Efficient Reasoningcited by 70
  3. 2025Unlocking Efficient Long-to-Short LLM Reasoning with Model Mergingcited by 58
  4. 2025Efficient Inference for Large Reasoning Models: A Surveycited by 58
  5. 2025InftyThink: Breaking the Length Limits of Long-Context Reasoning in Large Language Modelscited by 53
  6. 2025AdaCtrl: Towards Adaptive and Controllable Reasoning via Difficulty-Aware Budgetingcited by 50
  7. 2025Token Reduction Should Go Beyond Efficiency in Generative Models -- From Vision, Language to Multimodalitycited by 49
  8. 2026CRISP: Compressed Reasoning via Iterative Self-Policy Distillationcited by 49
  9. 2025Speculative Thinking: Enhancing Small-Model Reasoning with Large Model Guidance at Inference Timecited by 48
  10. 2025SelfBudgeter: Adaptive Token Allocation for Efficient LLM Reasoningcited by 47
  11. 2025Towards Agentic RAG with Deep Reasoning: A Survey of RAG-Reasoning Systems in LLMscited by 40
  12. 2025Answer Convergence as a Signal for Early Stopping in Reasoningcited by 38
  13. 2025ThinkSwitcher: When to Think Hard, When to Think Fastcited by 37
  14. 2025Making Slow Thinking Faster: Compressing LLM Chain-of-Thought via Step Entropycited by 37
  15. 2025Don't Overthink It: A Survey of Efficient R1-style Large Reasoning Modelscited by 36
  16. 2025Harnessing the Reasoning Economy: A Survey of Efficient Reasoning for Large Language Modelscited by 35
  17. 2025DLER: Doing Length pEnalty Right - Incentivizing More Intelligence per Token via Reinforcement Learningcited by 34
  18. 2025ConCISE: Confidence-guided Compression in Step-by-step Efficient Reasoningcited by 33
  19. 2025Pruning the Unsurprising: Efficient LLM Reasoning via First-Token Surprisalcited by 32
  20. 2025Reasoning on a Budget: A Survey of Adaptive and Controllable Test-Time Compute in LLMscited by 31
  21. 2025R1-Compress: Long Chain-of-Thought Compression via Chunk Compression and Searchcited by 29
  22. 2025FlashThink: An Early Exit Method For Efficient Reasoningcited by 28
  23. 2025PixelThink: Towards Efficient Chain-of-Pixel Reasoningcited by 28
  24. 2025Metacognitive Reuse: Turning Recurring LLM Reasoning Into Concise Behaviorscited by 28
  25. 2025Let LRMs Break Free from Overthinking via Self-Braking Tuningcited by 27
  26. 2025BudgetThinker: Empowering Budget-aware LLM Reasoning with Control Tokenscited by 27
  27. 2025Shorten After You're Right: Lazy Length Penalties for Reasoning RLcited by 25
  28. 2025VeriThinker: Learning to Verify Makes Reasoning Model Efficientcited by 24
  29. 2025Thought Manipulation: External Thought Can Be Efficient for Large Reasoning Modelscited by 23
  30. 2025Activation Steering for Chain-of-Thought Compressioncited by 22
  31. 2025A Survey of Token Compression for Efficient Multimodal Large Language Modelscited by 22
  32. 2025A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Modelscited by 22
  33. 2026Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Modelscited by 22
  34. 2025Incentivizing Dual Process Thinking for Efficient Large Language Model Reasoningcited by 21
  35. 2025ThinKV: Thought-Adaptive KV Cache Compression for Efficient Reasoning Modelscited by 21
  36. 2026Chain-of-Thought Compression Should Not Be Blind: V-Skip for Efficient Multimodal Reasoning via Dual-Path Anchoringcited by 21
  37. 2025Thinking Fast and Right: Balancing Accuracy and Reasoning Length with Adaptive Rewardscited by 20
  38. 2025Train Long, Think Short: Curriculum Learning for Efficient Reasoningcited by 20

50 most cited of 246 All 246 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Expediting and Elevating Large Language Model Reasoning via Hidden Chain-of-Thought Decoding
  2. 2024Token-Budget-Aware LLM Reasoning
  3. 2026Chain Of Thought Compression: A Theoretical Analysis
  4. 2026Zipping the Thought: When and How Compressed Reasoning Data Works in LLM Post-Training
  5. 2024Extending Token Computation for LLM Reasoning
  6. 2025Token Signature: Predicting Chain-of-Thought Gains with Token Decoding Feature in Large Language Models
  7. 2025How Well do LLMs Compress Their Own Chain-of-Thought? A Token Complexity Approach
  8. 2025SoftCoT: Soft Chain-of-Thought for Efficient Reasoning with LLMs