mlsanityData to 7 Oct 2026

Fast KV Compaction via Attention Matching

arXiv 2602.16284 ↗First posted 18 Feb 2026cs.LGcited by 28

Not on the map. Why?

Abstract (arXiv)

Scaling language models to long contexts is often bottlenecked by the size of the key-value (KV) cache. In deployed settings, long contexts are typically managed through compaction in token space via summarization. However, summarization can be highly lossy, substantially harming downstream performance. Recent work on Cartridges has shown that it is possible to train highly compact KV caches in latent space that closely match full-context performance, but at the cost of slow and expensive end-to-end optimization. This work describes an approach for fast context compaction in latent space through Attention Matching, which constructs compact keys and values to reproduce attention outputs and preserve attention mass at a per-KV-head level. We show that this formulation naturally decomposes into simple subproblems, some of which admit efficient closed-form solutions. Within this framework, we develop a family of methods that significantly push the Pareto frontier of compaction time versus quality, achieving up to 50x compaction in seconds on some datasets with little quality loss.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). First posted on arXiv 18 Feb 2026.

Builds on

0 of 0 references.

No references here yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 28.

28 papers.

Sort:

By area:

  1. 2026Self-Compacting Language Model Agentscited by 11
  2. 2026Training Transformers for KV Cache Compressibilitycited by 6
  3. 2026Cartridges at Scale: Training Modular KV Caches over Large Document Collectionscited by 6
  4. 2026Nearly Optimal Attention Coresetscited by 4
  5. 2026Parallel Context Compaction for Long-Horizon LLM Agent Servingcited by 4
  6. 2026End-to-End Context Compression at Scalecited by 4
  7. 2026A Simple Plug-in for Improving Eviction-Based KV Cache Compressioncited by 3
  8. 2026Neural Garbage Collection: Learning to Forget while Learning to Reasoncited by 2
  9. 2026Practical Online KV Cache Compaction for LLM Agents: An Empirical Studycited by 2
  10. 2026Prefix Sliding for efficient test-time scalingcited by 2
  11. 2026Spark3R: Asymmetric Token Reduction Makes Fast Feed-Forward 3D Reconstructioncited by 1
  12. 2026LOCKS: Page-Local Compact Key Summaries for Efficient Long-Context Decodingcited by 1
16 more
  1. 2026Cartridges++: KV Cache Compression without Off-Context Derailmentcited by 1
  2. 2026ARC-KV: Amortizing Anchor Search for Reconstruction-Based KV Cache Compactioncited by 1
  3. 2026PatchKV: Weight-Space Compensation of KV Cachecited by 1
  4. 2026Self-Study Reconsidered: The Hidden Fragility of Learning from Self-Generated QAcited by 0
  5. 2026NELSSA: A GPU-PNM Heterogeneous System for Mixed-Length LLM Serving via Length-based Request Placementcited by 0
  6. 2026Thought-Aware KV Cache Compaction for Reasoning via Adaptive Attention Matchingcited by 0
  7. 2026Parser States Already Know: Structure-Conditioned KV Persistence for Structured Generationcited by 0
  8. 2026Where Should a Document Live: Context, Representations, or Parameters?cited by 0
  9. 2026When to Evict, Not What to Keep: Draft-Guided Eviction for Training-Free KV-Cache Compressioncited by 0
  10. 2026ReLMem: Learning Recurrent Memory for Longitudinal EHR Modelingcited by 0
  11. 2026KV-Kaizen: Learning Context-Adaptive Cache Compression Choicescited by 0
  12. 2026Persistent Context Graphs for Efficient Memory Compaction in LLM Agentscited by 0
  13. 2026The Optimization Landscape of Learning Compacted Context Modelscited by 0
  14. 2026AttSVD:Prompt-Adaptive Low-Rank KV Cache Compression via Attention-Guided SVDcited by 0
  15. 2026Visual Memory Attacks Can Persist Through The KV Cachecited by 0
  16. 2026Rehearse Everything, Remember Nothing: Attic-KV Rehearses What Will Be Readcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Still: Amortized KV Cache Compaction in a Single Forward Pass
  2. 2026SGD-KV: Summarization Guided KV Cache Compression
  3. 2024Eigen Attention: Attention in Low-Rank Space for KV Cache Compression
  4. 2025SALS: Sparse Attention in Latent Space for KV cache Compression
  5. 2026Moment-KV: Momentum-Based Decode-Time KV Cache Compression for Long Generation
  6. 2025Progressive Sparse Attention: Algorithm and System Co-design for Efficient Attention in LLM Serving
  7. 2026PuzzleKV: Page-Wise Low-Rank Decomposition for KV Cache Compression
  8. 2025Task-KV: Task-aware KV Cache Optimization via Semantic Differentiation of Attention Heads