mlsanityData to 14 Oct 2026

BeaconKV: Key-Value Cache Compression Guided by Beacon Queries for Efficient Large Reasoning Model Inference

arXiv 2609.04971 ↗First posted 4 Sep 2026cs.LG cs.CLcited by 0

On the map: LLM reasoning → Test-time compute and efficient thinking

also in 1 more
Abstract (arXiv)

Large Reasoning Models (LRMs) achieve superior problem-solving through extended Chain-of-Thought (CoT) generation, but the resulting key-value (KV) cache grows linearly with sequence length and creates severe memory bottlenecks, often exceeding GPU capacity for long reasoning traces. Existing KV cache compression methods rely on recent queries to estimate future token importance, implicitly assuming these serve as reliable proxies for future attention patterns. We demonstrate that this assumption fails in long-horizon reasoning: certain decoding steps generate Thought Revisiting Tokens (TRT) that re-attend to distant previous context, such as task-solving plans formulated early in the trace. Through systematic analysis, we discover that queries corresponding to the TRT cluster into a small number of similarity groups in the embedding space. Based on this insight, we propose BeaconKV, a training-free KV cache compression method that maintains beacon queries, compact representatives for each global query cluster, to anticipate which KV pairs will be revisited without storing the entire query history. Across four open-source LRMs and diverse reasoning benchmarks, BeaconKV generally outperforms existing compression methods, achieving up to 5.8× memory reduction while nearly preserving full cache accuracy and improving throughput by over 4.3×.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 27 of its 32 references that are in the map. First posted on arXiv 4 Sep 2026.

Builds on

27 of 32 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025LazyEviction: Lagged KV Eviction with Attention Pattern Observation for Efficient Long Reasoning19 Jun
  2. 2025Multipole Attention for Efficient Long Context Reasoning16 Jun
  3. 2025SeerAttention-R: Sparse Attention Adaptation for Long Reasoning10 Jun
  4. 2025Reasoning Path Compression: Compressing Generation Trajectories for Efficient LLM Reasoning20 May
  5. 2025Dynamic Early Exit in Reasoning Models22 Apr
  6. 2025SEAL: Steerable Reasoning Calibration of Large Language Models for Free7 Apr
  7. 2025InftyThink: Breaking the Length Limits of Long-Context Reasoning in Large Language Models9 Mar
  8. 2025LightThinker: Thinking Step-by-Step Compression21 Feb
  9. 2024Titans: Learning to Memorize at Test Time31 Dec

Other references · 18

  1. 2017Attention Is All You Needcited by 60,361
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2025Qwen3 Technical Reportcited by 8,820
  4. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  5. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  6. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  7. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 880
  8. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 832
  9. 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inferencecited by 501
  10. 2024MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attentioncited by 383
  11. 2024Transformers are Multi-State RNNscited by 120
  12. 2025KVzip: Query-Agnostic KV Cache Compression with Context Reconstructioncited by 95
6 more
  1. 2025KeyDiff: Key Similarity-Based KV Cache Eviction for Long-Context LLM Inference in Resource-Constrained Environmentscited by 51
  2. 2024InfiniPot: Infinite Context Processing on Memory-Constrained LLMscited by 27
  3. 2025Cache What Lasts: Token Retention for Memory-Bounded KV Cache in LLMscited by 19
  4. 2026Fast KVzip: Efficient and Accurate LLM Inference with Gated KV Evictioncited by 16
  5. 2025Rectified Sparse Attentioncited by 10
  6. 2026Tell Me What To Learn: Generalizing Neural Memory to be Controllable in Natural Languagecited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025ThinKV: Thought-Adaptive KV Cache Compression for Efficient Reasoning Models
  2. 2025R-KV: Redundancy-aware KV Cache Compression for Reasoning Models
  3. 2025Breadcrumbs Reasoning: Memory-Efficient Reasoning with Compression Beacons
  4. 2026KARA: Efficient Reasoning LLM Serving via Sliding-Window KV Cache Compression
  5. 2026LongFlow: Efficient KV Cache Compression for Reasoning Models
  6. 2025Semantic Integrity Matters: Benchmarking and Preserving High-Density Reasoning in KV Cache Compression
  7. 2025Hold Onto That Thought: Assessing KV Cache Compression On Reasoning
  8. 2026CASK: Core-Aware Selective KV Compression for Reasoning Traces