mlsanityData to 14 Oct 2026

Thinking as Compression: Your Reasoning Model is Secretly a Context Compressor

arXiv 2605.28713 ↗First posted 27 May 2026cs.AIcited by 0

On the map: LLM reasoning → Test-time compute and efficient thinking

also in 1 more
Abstract (arXiv)

Context compression aims to shorten long context inputs with minimal information loss for LLM inference acceleration. While existing methods have shown promise, they typically rely on complex compression modules or compression-specific training, leaving the intrinsic capabilities of LLMs underexplored. In contrast, this work reveals that a thinking model itself can naturally compress long contexts by organizing task-relevant information. We thus derive Thinking as Compression (TaC), a new compression paradigm that treats thinking itself as compressed context. Without relying on specific dedicated compressor, TaC directly prompts the thinking model to generate thinking traces as the shortened context, already outperforming most representative compression methods. Further, given that raw thinking output may struggle with budget control and shortcut behaviors, we introduce Thinking as Compression Constrained (TaC-C), leveraging a simple reward-driven optimization framework to elicit intrinsic thinking as compact and controllable compressed context. Experiments across four long-context QA benchmarks demonstrate that TaC-C consistently outperforms existing baselines. At 4x and 8x compression ratios, it surpasses the strongest competitor by 17.4% and 23.4% in average F1, and by 15.7% and 21.7% in average Exact Match Score (EM), respectively.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 20 of its 46 references that are in the map. First posted on arXiv 27 May 2026.

Builds on

20 of 46 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Competitive Programming with Large Reasoning Models3 Feb

Other references · 19

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  6. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  7. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  8. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,444
  9. 2025Group Sequence Policy Optimizationcited by 835
  10. 2025Group-in-Group Policy Optimization for LLM Agent Trainingcited by 497
  11. 2024Long-context LLMs Struggle with Long In-context Learningcited by 286
  12. 2023RECOMP: Improving Retrieval-Augmented LMs with Compression and Selective Augmentationcited by 256
7 more
  1. 2023In-context Autoencoder for Context Compression in a Large Language Modelcited by 199
  2. 2024Long Context Compression with Activation Beaconcited by 104
  3. 2025GMSA: Enhancing Context Compression via Group Merging and Layer Semantic Alignmentcited by 21
  4. 2024Lossless data compression by large modelscited by 20
  5. 2026COMI: Coarse-to-fine Context Compression via Marginal Information Gaincited by 13
  6. 2026Read As Human: Compressing Context via Parallelizable Close Reading and Skimmingcited by 10
  7. 2025No Mean Feat: Simple, Strong Baselines for Context Compressioncited by 7

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026ConPress: Learning Efficient Reasoning from Multi-Question Contextual Pressure
  2. 2025Think Right: Learning to Mitigate Under-Over Thinking via Adaptive, Attentive Compression
  3. 2025How Well do LLMs Compress Their Own Chain-of-Thought? A Token Complexity Approach
  4. 2026Compress-Distill: Reasoning Trace Compression for Efficient Knowledge Distillation
  5. 2025Optimizing Length Compression in Large Reasoning Models
  6. 2025LightThinker: Thinking Step-by-Step Compression
  7. 2025Hold Onto That Thought: Assessing KV Cache Compression On Reasoning
  8. 2026Reasoning as Compression: Unifying Budget Forcing via the Conditional Information Bottleneck