mlsanityData to 7 Oct 2026

CORE: COverage CAlibration and Evicted-Mass REdistribution for KV Cache

arXiv 2610.02235 ↗First posted 28 Sep 2026cs.AR cs.AIcited by 0

On the map: Efficient LLM inference and serving → KV cache optimization

Abstract (arXiv)

Long-context decoding is increasingly constrained by key--value (KV) cache memory and bandwidth. Existing fixed-budget compression methods typically separate retention from compensation, while a retention ranking specifies neither discarded attention mass nor the direction of induced output error. We start from an exact factorization: eviction error equals evicted attention mass times the directional gap between the evicted centroid and retained output, highlighting the importance of set-level coverage in retention and mass-preserving memory writing. We introduce CORE COverage Calibration and Evicted-Mass REdistribution for KV Cache, which distills an offline allocation combining query utility and log-determinant coverage into a lightweight cache-aware indexer. At inference, one calibrated distribution drives both channels: its Top-B ordering retains complementary KV states, while its excluded allocation mass and conditional weights parameterize latent-memory writes without a separate write-weight predictor or online log-determinant evaluation. Our analysis provides a four-term pre-compensation error certificate, characterizes non-additive coverage interactions, and establishes mass-independent write stability with hierarchical bounds through recurrent and query-adaptive normalization. Across three backbones, CORE exceeds the strongest RULER baseline by up to 3.78 points at 90% compression; LongBench and repeated-eviction evaluations further demonstrate strong effectiveness and decoding efficiency.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 47 of its 55 references that are in the map. First posted on arXiv 28 Sep 2026.

Builds on

47 of 55 references.

Closest ancestors · 28

Same direction, newest first

  1. 2026Trust the Mass: Forced Weights in KV-Cache Eviction25 Aug
  2. 2026Every Cache Entry Earns Its Place: Global Allocation of Resolution and Coverage for KV Cache Compression7 Aug
  3. 2026ResKV: Reconstructing Omitted Attention Contributions for Fixed-Budget KV Cache Compression31 Jul
  4. 2026Coverage-Driven KV Cache Eviction for Efficient and Improved Inference of LLM28 Jun
  5. 2026MomentKV: Closing the Directional Gap in KV Cache Eviction for Long-Context Inference1 Jun
  6. 2026IndexMem: Learned KV-Cache Eviction with Latent Memory for Long-Context LLM Inference25 May
  7. 2026Tensor Cache: Eviction-conditioned Associative Memory for Transformers21 May
  8. 2026Rethinking KV Cache Eviction via a Unified Information-Theoretic Objective28 Apr
  9. 2026LKV: End-to-End Learning of Head-wise Budgets and Token Selection for LLM KV Cache Eviction22 Apr
  10. 2026Learning to Evict from Key-Value Cache10 Feb
  11. 2025Cache What Lasts: Token Retention for Memory-Bounded KV Cache in LLMs3 Dec
  12. 2025Expected Attention: KV Cache Compression by Estimating Attention from Future Queries Distribution1 Oct
16 more
  1. 2025KeyDiff: Key Similarity-Based KV Cache Eviction for Long-Context LLM Inference in Resource-Constrained Environments21 Apr
  2. 2025RocketKV: Accelerating Long-Context LLM Inference via Two-Stage KV Cache Compression19 Feb
  3. 2024DynamicKV: Task-Aware Adaptive KV Cache Compression for Long Context LLMs19 Dec
  4. 2024SCOPE: Optimizing Key-Value Cache Compression in Long-context Generation18 Dec
  5. 2024ClusterKV: Manipulating LLM KV Cache in Semantic Space for Recallable Compression4 Dec
  6. 2024Locret: Enhancing Eviction in Long-Context LLM Inference with Trained Retaining Heads on Consumer-Grade Devices2 Oct
  7. 2024Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inference16 Jul
  8. 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference16 Jun
  9. 2024PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling4 Jun
  10. 2024Reducing Transformer Key-Value Cache Size with Cross-Layer Attention21 May
  11. 2024PyramidInfer: Pyramid KV Cache Compression for High-throughput LLM Inference21 May
  12. 2024SnapKV: LLM Knows What You are Looking for Before Generation22 Apr
  13. 2023SparQ Attention: Bandwidth-Efficient LLM Inference8 Dec
  14. 2023Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs3 Oct
  15. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models24 Jun
  16. 2023Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Time26 May

Other references · 19

  1. 2017Attention Is All You Needcited by 59,932
  2. 2025Qwen3 Technical Reportcited by 8,195
  3. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  4. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,378
  5. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,270
  6. 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 700
  7. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 604
  8. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 571
  9. 2022Efficiently Scaling Transformer Inferencecited by 529
  10. 2024LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitaskscited by 367
  11. 2025SWE-smith: Scaling Data for Software Engineering Agentscited by 255
  12. 2023LongLoRA: Efficient Fine-tuning of Long-Context Large Language Modelscited by 243
7 more
  1. 2025AIMO-2 Winning Solution: Building State-of-the-Art Mathematical Reasoning Models with OpenMathReasoning datasetcited by 165
  2. 2025R-KV: Redundancy-aware KV Cache Compression for Reasoning Modelscited by 52
  3. 2025LazyEviction: Lagged KV Eviction with Attention Pattern Observation for Efficient Long Reasoningcited by 17
  4. 2025Mixing Importance with Diversity: Joint Optimization for KV Cache Compression in Large Vision-Language Modelscited by 13
  5. 2026Value-Aware Stochastic KV Cache Eviction for Reasoning Modelscited by 8
  6. 2022Lazy and Fast Greedy MAP Inference for Determinantal Point Processcited by 5
  7. 2026Error Certificates for KV-Cache Eviction via Randomized Designcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026When Does Value-Aware KV Eviction Help? A Fixed-Contract Diagnostic for Non-Monotone Cache Compression
  2. 2026KV2: A Self-Refining KV Cache
  3. 2026Where Matters More Than What: Decoding-aligned KV Cache Compression via Position-aware Pseudo Queries
  4. 2026Behavior-Preserving KV Cache Compression
  5. 2026KVDiagnosis: A Diagnostic Benchmark for KV-Cache Compression in Long-Context Language Models
  6. 2026DeferKV: Rethinking Eviction Timing for One-Shot KV Cache Compression
  7. 2025LAVa: Layer-wise KV Cache Eviction with Dynamic Budget Allocation
  8. 2026The risk of KV cache compression