mlsanityData to 14 Oct 2026

LightThinker: Thinking Step-by-Step Compression

arXiv 2502.15589 ↗First posted 21 Feb 2025cs.CL cs.AI cs.IRcited by 104

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Large language models (LLMs) have shown remarkable performance in complex reasoning tasks, but their efficiency is hindered by the substantial memory and computational costs associated with generating lengthy tokens. In this paper, we propose LightThinker, a novel method that enables LLMs to dynamically compress intermediate thoughts during reasoning. Inspired by human cognitive processes, LightThinker compresses verbose thought steps into compact representations and discards the original reasoning chains, thereby significantly reducing the number of tokens stored in the context window. This is achieved by training the model on when and how to perform compression through data construction, mapping hidden states to condensed gist tokens, and creating specialized attention masks. Additionally, we introduce the Dependency (Dep) metric to quantify the degree of compression by measuring the reliance on historical tokens during generation. Extensive experiments on four datasets and two models show that LightThinker reduces peak memory usage and inference time, while maintaining competitive accuracy. Our work provides a new direction for improving the efficiency of LLMs in complex reasoning tasks without sacrificing performance. Code is released at https://github.com/zjunlp/LightThinker.

Key facts. Cited by 104 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 32 of its 53 references that are in the map. First posted on arXiv 21 Feb 2025.

Builds on

32 of 53 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruning22 Jan
  2. 2024Compressed Chain of Thought: Efficient Reasoning Through Dense Representations17 Dec
  3. 2024C3oT: Generating Shorter Chain-of-Thought without Compromising Effectiveness16 Dec
  4. 2024A Comparative Study on Reasoning Patterns of OpenAI's o1 Model17 Oct
  5. 2024Concise Thoughts: Impact of Output Length on LLM Reasoning and Cost29 Jul

Other references · 27

  1. 2017Attention Is All You Needcited by 60,361
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  5. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  6. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  7. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  8. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
  9. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  10. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 880
  11. 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 797
  12. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 637
15 more
  1. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 591
  2. 2024PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funnelingcited by 432
  3. 2023Learning to Compress Prompts with Gist Tokenscited by 390
  4. 2023Adapting Language Models to Compress Contextscited by 356
  5. 2022Reasoning with Language Model Prompting: A Surveycited by 252
  6. 2025Training Language Models to Reason Efficientlycited by 247
  7. 2024From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Stepcited by 235
  8. 2024Long Context Compression with Activation Beaconcited by 104
  9. 2020Thinking Fast and Slow in AIcited by 70
  10. 2024Can Language Models Learn to Skip Steps?cited by 59
  11. 2024SepLLM: Accelerate Large Language Models by Compressing One Segment into One Separatorcited by 41
  12. 2024Break the Chain: Large Language Models Can be Shortcut Reasonerscited by 32
  13. 2024SCOPE: Optimizing Key-Value Cache Compression in Long-context Generationcited by 20
  14. 2023ChatGPT is a Remarkable Tool -- For Expertscited by 16
  15. 2024OneGen: Efficient One-Pass Unified Generation and Retrieval for LLMscited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 62; 2026: 42.

104 papers.

Sort:

By area:

  1. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Modelscited by 464
  2. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 384
  3. 2025MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agentscited by 238
  4. 2025A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?cited by 223
  5. 2025DAST: Difficulty-Adaptive Slow-Thinking for Large Reasoning Modelscited by 158
  6. 2025ACON: Optimizing Context Compression for Long-horizon LLM Agentscited by 136
  7. 2025A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyondcited by 132
  8. 2025Efficient Reasoning Models: A Surveycited by 95
  9. 2025AdaCoT: Pareto-Optimal Adaptive Chain-of-Thought Triggering via Reinforcement Learningcited by 87
  10. 2025A Survey on Latent Reasoningcited by 81
  11. 2025Efficient Inference for Large Reasoning Models: A Surveycited by 61
  12. 2025A Survey of LLM-Driven AI Agent Communication: Protocols, Security Risks, and Defense Countermeasurescited by 57
38 more
  1. 2025InftyThink: Breaking the Length Limits of Long-Context Reasoning in Large Language Modelscited by 54
  2. 2025Token Reduction Should Go Beyond Efficiency in Generative Models -- From Vision, Language to Multimodalitycited by 50
  3. 2025Speculative Thinking: Enhancing Small-Model Reasoning with Large Model Guidance at Inference Timecited by 48
  4. 2025SSR: Enhancing Depth Perception in Vision-Language Models via Rationale-Guided Spatial Reasoningcited by 46
  5. 2025Implicit Reasoning in Large Language Models: A Comprehensive Surveycited by 43
  6. 2025Towards Agentic RAG with Deep Reasoning: A Survey of RAG-Reasoning Systems in LLMscited by 42
  7. 2025Harnessing the Reasoning Economy: A Survey of Efficient Reasoning for Large Language Modelscited by 36
  8. 2025Retro-Search: Exploring Untaken Paths for Deeper and Efficient Reasoningcited by 34
  9. 2025ConCISE: Confidence-guided Compression in Step-by-step Efficient Reasoningcited by 33
  10. 2025Reasoning on a Budget: A Survey of Adaptive and Controllable Test-Time Compute in LLMscited by 31
  11. 2025R1-Compress: Long Chain-of-Thought Compression via Chunk Compression and Searchcited by 30
  12. 2025FlashThink: An Early Exit Method For Efficient Reasoningcited by 28
  13. 2025Promoting Efficient Reasoning with Verifiable Stepwise Rewardcited by 28
  14. 2025Let LRMs Break Free from Overthinking via Self-Braking Tuningcited by 27
  15. 2025ThinKV: Thought-Adaptive KV Cache Compression for Efficient Reasoning Modelscited by 21
  16. 2025SofT-GRPO: Surpassing Discrete-Token LLM Reinforcement Learning via Gumbel-Reparameterized Soft-Thinking Policy Optimizationcited by 21
  17. 2026Stabilizing Recurrent Dynamics for Test-Time Scalable Latent Reasoning in Looped Language Modelscited by 21
  18. 2026Render-of-Thought: Rendering Textual Chain-of-Thought as Images for Visual Latent Reasoningcited by 20
  19. 2025A Survey of Slow Thinking-based Reasoning LLMs using Reinforced Learning and Inference-time Scaling Lawcited by 19
  20. 2025Can Pruning Improve Reasoning? Revisiting Long-CoT Compression with Capability in Mind for Better Reasoningcited by 19
  21. 2025Towards Concise and Adaptive Thinking in Large Reasoning Models: A Surveycited by 18
  22. 2026MEMENTO: Teaching LLMs to Manage Their Own Contextcited by 18
  23. 2025Reasoning Path Compression: Compressing Generation Trajectories for Efficient LLM Reasoningcited by 17
  24. 2025ThinkLess: A Training-Free Inference-Efficient Method for Reducing Reasoning Redundancycited by 17
  25. 2025Not All Thoughts are Generated Equal: Efficient LLM Reasoning via Multi-Turn Reinforcement Learningcited by 16
  26. 2025Two Experts Are All You Need for Steering Thinking: Reinforcing Cognitive Effort in MoE Reasoning Models Without Additional Trainingcited by 16
  27. 2025LazyEviction: Lagged KV Eviction with Attention Pattern Observation for Efficient Long Reasoningcited by 16
  28. 2026Thinking in Streaming Videocited by 16
  29. 2025Generative AI Act II: Test Time Scaling Drives Cognition Engineeringcited by 15
  30. 2025Concise Reasoning, Big Gains: Pruning Long Reasoning Trace with Difficulty-Aware Promptingcited by 15
  31. 2025StreamingThinker: Large Language Models Can Think While Readingcited by 15
  32. 2025CoIn: Counting the Invisible Reasoning Tokens in Commercial Opaque LLM APIscited by 14
  33. 2025TrimR: Verifier-based Training-Free Thinking Compression for Efficient Test-Time Scalingcited by 14
  34. 2025Reasoning Model Unlearning: Forgetting Traces, Not Just Answers, While Preserving Reasoning Skillscited by 14
  35. 2025Exploring the System 1 Thinking Capability of Large Reasoning Modelscited by 10
  36. 2025Not All Tokens Matter: Towards Efficient LLM Reasoning via Token Significance in Reinforcement Learningcited by 10
  37. 2025A*-Thought: Efficient Reasoning via Bidirectional Compression for Low-Resource Settingscited by 9
  38. 2025Demystifying Hybrid Thinking: Can LLMs Truly Switch Between Think and No-Think?cited by 9

50 most cited of 104 All 104 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026LightThinker++: From Reasoning Compression to Memory Management
  2. 2026Shorthand for Thought: Compressing LLM Reasoning via Entropy-Guided Supertokens
  3. 2025Making Slow Thinking Faster: Compressing LLM Chain-of-Thought via Step Entropy
  4. 2025How Well do LLMs Compress Their Own Chain-of-Thought? A Token Complexity Approach
  5. 2026Memory Augmentation Unlocks Efficient Chain-of-Thought Reasoning
  6. 2025Activation Steering for Chain-of-Thought Compression
  7. 2025Efficient Reasoning with Hidden Thinking
  8. 2025Think Silently, Think Fast: Dynamic Latent Compression of LLM Reasoning Chains