mlsanityData to 14 Oct 2026

REAL: REtrieval-reAsoning and Logic-constructed Attention Behaviors for Long-Context KV Cache Compression

arXiv 2508.15806 ↗First posted 14 Aug 2025cs.CL cs.AIcited by 2

On the map: LLM reasoning → Test-time compute and efficient thinking

also in 1 more
Abstract (arXiv)

The growing sequence length of large language models poses significant challenges for key-value (KV) caches. Existing state-of-the-art cache eviction methods primarily analyze the inference behavior of attention heads in successful retrieval-reasoning cases, often overlooking diverse behaviors in failure cases, such as bias and distraction. This oversight limits the potential to leverage heterogeneous head behaviors for improved eviction performance. Inspired by the confusion matrix, we introduce an Attention Behavior Matrix to comprehensively analyze attention head behaviors in both success and failure scenarios. By maximizing the signal-to-noise ratio -- strengthening valid reasoning pathways in success cases while inhibiting noise from bias and distraction in failure cases -- we propose REtrieval-reAsoning and Logic-constructed (REAL) KV cache eviction, the first method to leverage multi-behavior analysis. Comprehensive evaluations show that REAL achieves remarkable performance across various models and benchmarks; notably, on LongBench v2, it achieves comparable accuracy to the strongest baseline, HeadKV-R2, while requiring 32x less space (Figure 1). By offering a novel perspective on behavior analysis, we pave the way for a shift from success-only to comprehensive, failure-aware methods in long-context modeling. Our code is available at https://github.com/yonseicasl/REAL.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 18 of its 25 references that are in the map. First posted on arXiv 14 Aug 2025.

Builds on

18 of 25 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Not All Heads Matter: A Head-Level KV Cache Compression Method with Integrated Retrieval and Reasoning25 Oct

Other references · 17

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  4. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 880
  5. 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 526
  6. 2023Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Timecited by 439
  7. 2024PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funnelingcited by 432
  8. 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 414
  9. 2024MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attentioncited by 383
  10. 2024LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitaskscited by 379
  11. 2023SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compressioncited by 367
  12. 2024DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Headscited by 288
5 more
  1. 2024Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inferencecited by 235
  2. 2024RetrievalAttention: Accelerating Long-Context LLM Inference via Vector Retrievalcited by 116
  3. 2024RazorAttention: Efficient KV Cache Compression Through Retrieval Headscited by 100
  4. 2023OliVe: Accelerating Large Language Models via Hardware-friendly Outlier-Victim Pair Quantizationcited by 92
  5. 2023ZeroQuant-FP: A Leap Forward in LLMs Post-Training W4A8 Quantization Using Floating-Point Formatscited by 44

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Random Attention: Rethinking KV Cache Eviction for Efficient Reasoningcited by 1
  2. 2026GrowPage: On-Demand KV Budgeting for Efficient LLM Reasoning Servingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Behavior-Preserving KV Cache Compression
  2. 2025G-KV: Decoding-Time KV Cache Eviction with Global Attention
  3. 2025Hold Onto That Thought: Assessing KV Cache Compression On Reasoning
  4. 2026Make Each Token Count: Towards Improving Long-Context Performance with KV Cache Eviction
  5. 2025LLMs Know What to Drop: Self-Attention Guided KV Cache Eviction for Efficient Long-Context Inference
  6. 2025CompressKV: Semantic Retrieval Heads Know What Tokens are Not Important Before Generation
  7. 2026Reformulating KV Cache Eviction Problem for Long-Context LLM Inference
  8. 2025LazyEviction: Lagged KV Eviction with Attention Pattern Observation for Efficient Long Reasoning