mlsanityData to 14 Oct 2026

ThinKV: Thought-Adaptive KV Cache Compression for Efficient Reasoning Models

arXiv 2510.01290 ↗First posted 1 Oct 2025cs.LGcited by 21

On the map: LLM reasoning → Test-time compute and efficient thinking

also in 1 more
Abstract (arXiv)

The long-output context generation of large reasoning models enables extended chain of thought (CoT) but also drives rapid growth of the key-value (KV) cache, quickly overwhelming GPU memory. To address this challenge, we propose ThinKV, a thought-adaptive KV cache compression framework. ThinKV is based on the observation that attention sparsity reveals distinct thought types with varying importance within the CoT. It applies a hybrid quantization-eviction strategy, assigning token precision by thought importance and progressively evicting tokens from less critical thoughts as reasoning trajectories evolve. Furthermore, to implement ThinKV, we design a kernel that extends PagedAttention to enable efficient reuse of evicted tokens' memory slots, eliminating compaction overheads. Extensive experiments on DeepSeek-R1-Distill, GPT-OSS, and NVIDIA AceReason across mathematics and coding benchmarks show that ThinKV achieves near-lossless accuracy with less than 5% of the original KV cache, while improving performance with up to 5.8x higher inference throughput over state-of-the-art baselines.

Key facts. Cited by 21 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 50 of its 76 references that are in the map. First posted on arXiv 1 Oct 2025.

Builds on

50 of 76 references.

Closest ancestors · 17

Same direction, newest first

  1. 2025gpt-oss-120b & gpt-oss-20b Model Card8 Aug
  2. 2025ReasonCache: Accelerating Large Reasoning Model Serving through KV Cache Sharing29 Jul
  3. 2025Understanding Reasoning in Thinking Language Models via Steering Vectors22 Jun
  4. 2025LazyEviction: Lagged KV Eviction with Attention Pattern Observation for Efficient Long Reasoning19 Jun
  5. 2025Multipole Attention for Efficient Long Context Reasoning16 Jun
  6. 2025R-KV: Redundancy-aware KV Cache Compression for Reasoning Models30 May
  7. 2025PM-KVQ: Progressive Mixed-precision KV Cache Quantization for Long-CoT LLMs24 May
  8. 2025Reasoning Path Compression: Compressing Generation Trajectories for Efficient LLM Reasoning20 May
  9. 2025SEAL: Steerable Reasoning Calibration of Large Language Models for Free7 Apr
  10. 2025InftyThink: Breaking the Length Limits of Long-Context Reasoning in Large Language Models9 Mar
  11. 2025LightThinker: Thinking Step-by-Step Compression21 Feb
  12. 2025TokenSkip: Controllable Chain-of-Thought Compression in LLMs17 Feb
5 more
  1. 2025RaaS: Reasoning-Aware Attention Sparsity for Efficient LLM Reasoning16 Feb
  2. 2025s1: Simple test-time scaling31 Jan
  3. 2024Token-Budget-Aware LLM Reasoning24 Dec
  4. 2024Compressed Chain of Thought: Efficient Reasoning Through Dense Representations17 Dec
  5. 2024Not All Heads Matter: A Head-Level KV Cache Compression Method with Integrated Retrieval and Reasoning25 Oct

Other references · 33

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2025Qwen3 Technical Reportcited by 8,820
  3. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  4. 2023Let's Verify Step by Stepcited by 3,789
  5. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  6. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  7. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  8. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 637
  9. 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inferencecited by 501
  10. 2024PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funnelingcited by 432
  11. 2025Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attentioncited by 407
  12. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 320
21 more
  1. 2024Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inferencecited by 235
  2. 2024GEAR: An Efficient KV Cache Compression Recipe for Near-Lossless Generative Inference of LLMcited by 158
  3. 2024LongWriter: Unleashing 10,000+ Word Generation from Long Context LLMscited by 144
  4. 2024ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inferencecited by 134
  5. 2025Thought Anchors: Which LLM Reasoning Steps Matter?cited by 129
  6. 2025Sketch-of-Thought: Efficient LLM Reasoning with Adaptive Cognitive-Inspired Sketchingcited by 117
  7. 2024Dynamic Memory Compression: Retrofitting LLMs for Accelerated Inferencecited by 116
  8. 2025SoftCoT: Soft Chain-of-Thought for Efficient Reasoning with LLMscited by 116
  9. 2024Model Tells You Where to Merge: Adaptive KV Cache Merging for LLMs on Long-Context Taskscited by 96
  10. 2025KVzip: Query-Agnostic KV Cache Compression with Context Reconstructioncited by 95
  11. 2025Paper2Code: Automating Code Generation from Scientific Papers in Machine Learningcited by 70
  12. 2024Palu: Compressing KV-Cache with Low-Rank Projectioncited by 64
  13. 2024LoRC: Low-Rank Compression for LLMs KV Cache with a Progressive Compression Strategycited by 51
  14. 2024NACL: A General and Effective KV Cache Eviction Framework for LLMs at Inference Timecited by 43
  15. 2025Mind the Memory Gap: Unveiling GPU Bottlenecks in Large-Batch LLM Inferencecited by 33
  16. 2025Chain-of-Thought Matters: Improving Long-Context Language Models with Reasoning Path Supervisioncited by 21
  17. 2025Dialogue Without Limits: Constant-Sized KV Caches for Extended Responses in LLMscited by 20
  18. 2025LaCache: Ladder-Shaped KV Caching for Efficient Long-Context Modeling of Large Language Modelscited by 9
  19. 2025A2ATS: Retrieval-Based KV Cache Reduction via Windowed Rotary Position Embedding and Query-Aware Vector Quantizationcited by 7
  20. 2025CalibQuant: 1-Bit KV Cache Quantization for Multimodal LLMscited by 3
  21. 2025Accelerating LLM Inference with Flexible N:M Sparsity via A Fully Digital Compute-in-Memory Acceleratorcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 20.

21 papers.

Sort:

By area:

  1. 2026MEMENTO: Teaching LLMs to Manage Their Own Contextcited by 18
  2. 2026SideQuest: Model-Driven KV Cache Management for Long-Horizon Agentic Reasoningcited by 11
  3. 2026OSCAR: Offline Spectral Covariance-Aware Rotation for 2-bit KV Cache Quantizationcited by 9
  4. 2025SkipKV: Selective Skipping of KV Generation and Storage for Efficient Inference with Large Reasoning Modelscited by 5
  5. 2026ArborKV: Structure-Aware KV Cache Management for Scaling Tree-based LLM Reasoningcited by 4
  6. 2026TriAxialKV: Toward Extreme Low-Precision KV-Cache Quantization for Agentic Inference Taskscited by 3
  7. 2026Epiphany-Aware KV Cache Eviction Without the Attention Matrixcited by 2
  8. 2026KARA: Efficient Reasoning LLM Serving via Sliding-Window KV Cache Compressioncited by 2
  9. 2026Prefix Sliding for efficient test-time scalingcited by 2
  10. 2026Adaptive Mass-Segmented KV Compression for Long-Context Reasoningcited by 1
  11. 2026Polestar: Drift-Aware Cache Calibration and Token Commitment for Efficient Inference of Diffusion LLMscited by 1
  12. 2026Language Models Can Control Their Own Attentioncited by 1
9 more
  1. 2026Monte Carlo Estimation for KV Cache Evictioncited by 1
  2. 2026State commitment learning: training language models to distinguish computation from memorycited by 0
  3. 2026Memory Augmentation Unlocks Efficient Chain-of-Thought Reasoningcited by 0
  4. 2026GrowPage: On-Demand KV Budgeting for Efficient LLM Reasoning Servingcited by 0
  5. 2026StepKV: Step-Aware KV Cache Compression for LLM Agentscited by 0
  6. 2026ActKV: Efficient LLM Agents through Action-Guided KV Cache Managementcited by 0
  7. 2026Does Execution Require Target KV Fidelity? A Mixed-Fidelity KV Runtime for LLM Servingcited by 0
  8. 2026WakeKV: Reactive, Reversible KV Residency for Heads That Change Their Mindscited by 0
  9. 2026Spend Bytes on Breadth: Precision-Count Trade-offs for Decode-Time KV Compression in Long Chain-of-Thought Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Thought-Aware KV Cache Compaction for Reasoning via Adaptive Attention Matching
  2. 2026BeaconKV: Key-Value Cache Compression Guided by Beacon Queries for Efficient Large Reasoning Model Inference
  3. 2026LongFlow: Efficient KV Cache Compression for Reasoning Models
  4. 2025Semantic Integrity Matters: Benchmarking and Preserving High-Density Reasoning in KV Cache Compression
  5. 2025Hold Onto That Thought: Assessing KV Cache Compression On Reasoning
  6. 2025Lethe: Layer- and Time-Adaptive KV Cache Pruning for Reasoning-Intensive LLM Serving
  7. 2026Value-Aware Stochastic KV Cache Eviction for Reasoning Models
  8. 2026Random Attention: Rethinking KV Cache Eviction for Efficient Reasoning