mlsanityData to 7 Oct 2026

VestigeKV: The NoPE-MLA KV Cache Carries Its Own Sparse-Attention Signal in a Vestigial Branch

arXiv 2609.03949 ↗First posted 3 Sep 2026cs.LG cs.CLcited by 0

On the map: Efficient LLM inference and serving → KV cache optimization

Abstract (arXiv)

A long-lived KV cache must be compressed before the queries that will read it exist. Selection by observed attention collapses there: on a NoPE-MLA model, H2O and SnapKV retrieve 0.00 and 0.33 of needles at 8x compression, because a token's importance has not yet been observed. VestigeKV instead derives a sparse attention pattern from a signal the cache already carries, occupying the sparse-attention literature's one unoccupied quadrant: training-free and query-independent. In NoPE-MLA the 64-dimensional decoupled branch is a vestige of RoPE that training repurposes into a salience channel; reading 11% of each row, it partitions the cache into an attended tier and a GPU-resident archive that no row ever leaves, reachable each step by a certified, query-adaptive trigger. Nothing is trained and cache rows are never quantized, so every quality effect attributes to selection and scheduling. On Kimi Linear 48B, retrieval holds at 1.00 under 8x and 0.96 under 32x from 8k to 65k context, with zero gap to full-row selection, and the recall tier holds 128x at 1.00 (8k). Both tiers stay on the GPU, so the win is speed, not memory: the per-step scan reads ~26% of the bytes dense attention would, and on a two-node sglang deployment the crossover sits at ~40k context, reaching 1.18x at 256k and 1.39x at 496k. The mechanism is exclusive to NoPE: the identical operator on a RoPE MLA collapses to 0.08, query-independent salience exists only without rotation, and query-universal exact merging is provably impossible under RoPE. All thresholds were frozen before their data; 20 archived verdicts and 8 closed routes accompany the paper.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 16 of its 19 references that are in the map. First posted on arXiv 3 Sep 2026.

Builds on

16 of 19 references.

Closest ancestors · 5

Same direction, newest first

  1. 2026Training Transformers for KV Cache Compressibility7 May
  2. 2025KeepKV: Achieving Periodic Lossless KV Cache Compression for Efficient LLM Inference14 Apr
  3. 2024MatryoshkaKV: Adaptive KV Compression via Trainable Orthogonal Projection16 Oct
  4. 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference16 Jun
  5. 2024SnapKV: LLM Knows What You are Looking for Before Generation22 Apr

Other references · 11

  1. 2020Longformer: The Long-Document Transformercited by 3,362
  2. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  3. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,093
  4. 2025Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attentioncited by 391
  5. 2023The Impact of Positional Encoding on Length Generalization in Transformerscited by 319
  6. 2025MoBA: Mixture of Block Attention for Long-Context LLMscited by 188
  7. 2025Kimi Linear: An Expressive, Efficient Attention Architecturecited by 169
  8. 2024Transformers are Multi-State RNNscited by 120
  9. 2024Length Generalization of Causal Transformers without Position Encodingcited by 46
  10. 2026Irminsul: MLA-Native Position-Independent Caching for Agentic LLM Servingcited by 3
  11. 2026Kamera: Unified Position-Invariant Multimodal KV Cache for Training-Free Reusecited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025SALS: Sparse Attention in Latent Space for KV cache Compression
  2. 2026Trust the Mass: Forced Weights in KV-Cache Eviction
  3. 2026Sparse Attention as a Range Searching Problem: Towards an Inference-Efficient Index for KV Cache
  4. 2026Beyond Mean Attention: Diversity-Aware, Layer-Wise Scoring for KV Cache Eviction
  5. 2025Expected Attention: KV Cache Compression by Estimating Attention from Future Queries Distribution
  6. 2026Beyond Sparse Weights: When Is Attention Compressible?
  7. 2026ResKV: Reconstructing Omitted Attention Contributions for Fixed-Budget KV Cache Compression
  8. 2025Sparse Attention across Multiple-context KV Cache