mlsanityData to 7 Oct 2026

ManifoldKV: Training-Free KV Cache Compression via Euclidean Outlier Detection

arXiv 2602.08343 ↗First posted 9 Feb 2026cs.LG cs.AI cs.CLcited by 2

On the map: Efficient LLM inference and serving → KV cache optimization

Abstract (arXiv)

Long-context inference is constrained by KV-cache memory, which grows linearly with sequence length; KV-cache compression therefore hinges on reliably selecting which past tokens to retain. Most geometry-based eviction methods score keys by cosine similarity to a global centroid, but cosine is scale-invariant and can discard magnitude cues that distinguish semantically salient tokens. We propose ManifoldKV, a training-free scorer that ranks tokens by Euclidean distance to the key centroid, capturing both angular and radial deviations. On the RULER benchmark, ManifoldKV achieves 95.7% accuracy at 4K-16K contexts with 20% compression; matching the best geometric baseline while improving robustness in two regimes where cosine scoring fails. First, on multi-key retrieval, ManifoldKV reduces directional collisions, achieving 92.4% vs KeyDiff's 77.0% (+15.4 points) on 3-key NIAH at 50% compression. Second, to address dilution and performance collapse of global centroids at 64K context, we introduce WindowedManifoldKV, which restores accuracy to 84.3% at 25% compression, a 49-point recovery over global L2 and +3.2 points over KeyDiff. The method requires only 3 lines of code and works across 4 architectures without tuning.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 18 of its 24 references that are in the map. First posted on arXiv 9 Feb 2026.

Builds on

18 of 24 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025KeyDiff: Key Similarity-Based KV Cache Eviction for Long-Context LLM Inference in Resource-Constrained Environments21 Apr
  2. 2025CriticalKV: Optimizing KV Cache Eviction from an Output Perturbation Perspective6 Feb
  3. 2024A Survey on Large Language Model Acceleration based on KV Cache Management27 Dec
  4. 2024DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads14 Oct
  5. 2024Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inference16 Jul
  6. 2024A Simple and Effective L2 Norm-Based Strategy for KV Cache Compression17 Jun
  7. 2024SnapKV: LLM Knows What You are Looking for Before Generation22 Apr
  8. 2023Efficient Memory Management for Large Language Model Serving with PagedAttention12 Sep
  9. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models24 Jun

Other references · 9

  1. 2017Attention Is All You Needcited by 59,932
  2. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  3. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,270
  4. 2020Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuningcited by 658
  5. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 604
  6. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 571
  7. 2022Efficiently Scaling Transformer Inferencecited by 529
  8. 2019Intrinsic dimension of data representations in deep neural networkscited by 276
  9. 2018Estimating the intrinsic dimension of datasets by a minimal neighborhood informationcited by 246

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026ECOKV: Geometry-Aware KV Cache Eviction via Complementary Diversity Metricscited by 0
  2. 2026ValueDiff: Value-Geometric KV Cache Eviction for Sink-Suppressed LLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026AnchorKV: Anchor-Residual KV Cache Compression
  2. 2026Make Each Token Count: Towards Improving Long-Context Performance with KV Cache Eviction
  3. 2025GraphKV: Breaking the Static Selection Paradigm with Graph-Based KV Cache Eviction
  4. 2026GRKV: Global Regression for Training-Free KV Cache Compression in Long-Context LLMs
  5. 2026Beyond Token Eviction: Mixed-Dimension Budget Allocation for Efficient KV Cache Compression
  6. 2025LagKV: Lag-Relative Information of the KV Cache Tells Which Tokens Are Important
  7. 2025Compactor: Calibrated Query-Agnostic KV Cache Compression with Approximate Leverage Scores
  8. 2026ParisKV: Fast and Drift-Robust KV-Cache Retrieval for Long-Context LLMs