mlsanityData to 13 Oct 2026

CompilerKV: Risk-Adaptive KV Compression via Offline Experience Compilation

arXiv 2602.08686 ↗First posted 9 Feb 2026cs.LG cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

Prefill-only KV compression freezes a token subset at the end of prefill and decodes from it without further eviction. The retention decision is therefore irreversible, yet existing methods estimate the corrective signals it relies on, per-head reliability and prompt-level compression sensitivity, online from a single noisy prompt. We argue this is the wrong statistical unit: these signals exhibit far higher cross-prompt regularity than within-prompt signal-to-noise. We introduce textscCompilerKV, a KV-retention policy whose corrective tables are compiled offline from a calibration corpus, reducing online correction after the standard observation-window scan to O(1) lookups plus a budget clamp. We find that compiled retention tables behave as portable architectural priors: rankings transfer across disjoint corpora on four backbones (mean Spearman barρ=0.90), and direct model-to-model table transfer costs only 0.4--0.8 LongBench points on average. At a 512-token budget, textscCompilerKV attains compressed-SOTA on all four backbones, improving over the strongest prefill-only baseline by +1.67 points on average (task-bootstrap 95% CI [+1.08,+2.37]). Pressure regimes amplify the gap: under a fixed 512/32k cache ratio, CompilerKV remains the strongest compressed method through 128k RULER (~!73 vs. FullKV ~!79, SnapKV ~!38); on 32k NIAH it reaches 0.89 vs. SnapKV 0.42; and at 32k input, retaining only 1.56% of the prefill KV, batch-16 serving remains feasible where FullKV is OOM.

Key facts. Cited by 2 papers in the map (data through 13 Oct 2026). Builds on 39 of its 44 references that are in the map. First posted on arXiv 9 Feb 2026.

Builds on

39 of 44 references.

References · 39

  1. 2017Attention Is All You Needcited by 60,106
  2. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,716
  3. 2023Let's Verify Step by Stepcited by 3,672
  4. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,137
  5. 2019Generating Long Sequences with Sparse Transformerscited by 1,538
  6. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,386
  7. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,302
  8. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 856
  9. 2019Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Prunedcited by 849
  10. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 813
  11. 2017The NarrativeQA Reading Comprehension Challengecited by 592
  12. 2019SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarizationcited by 533
27 more
  1. 2022Efficiently Scaling Transformer Inferencecited by 532
  2. 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inferencecited by 489
  3. 2021A Dataset of Information-Seeking Questions and Answers Anchored in Research Paperscited by 478
  4. 2020Efficient Content-Based Sparse Attention with Routing Transformerscited by 470
  5. 2019Multi-News: a Large-Scale Multi-Document Summarization Dataset and Abstractive Hierarchical Modelcited by 429
  6. 2024PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funnelingcited by 422
  7. 2023RepoBench: Benchmarking Repository-Level Code Auto-Completion Systemscited by 350
  8. 2021Efficient Attentions for Long Document Summarizationcited by 342
  9. 2024Titans: Learning to Memorize at Test Timecited by 312
  10. 2024DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Headscited by 284
  11. 2023Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Timecited by 250
  12. 2024Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inferencecited by 232
  13. 2023In-context Autoencoder for Context Compression in a Large Language Modelcited by 194
  14. 2024ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inferencecited by 131
  15. 2023SparQ Attention: Bandwidth-Efficient LLM Inferencecited by 118
  16. 2023LongCoder: A Long-Range Pre-trained Language Model for Code Completioncited by 111
  17. 2024Not All Heads Matter: A Head-Level KV Cache Compression Method with Integrated Retrieval and Reasoningcited by 95
  18. 2025KVzip: Query-Agnostic KV Cache Compression with Context Reconstructioncited by 91
  19. 2024Attention Score is not All You Need for Token Importance Indicator in KV Cache Reduction: Value Also Matterscited by 72
  20. 2025CAKE: Cascading and Adaptive KV Cache Eviction with Layer Preferencescited by 58
  21. 2025ChunkKV: Semantic-Preserving KV Cache Compression for Efficient Long-Context LLM Inferencecited by 57
  22. 2024DynamicKV: Task-Aware Adaptive KV Cache Compression for Long Context LLMscited by 42
  23. 2025RocketKV: Accelerating Long-Context LLM Inference via Two-Stage KV Cache Compressioncited by 41
  24. 2025Which Heads Matter for Reasoning? RL-Guided KV Cache Compressioncited by 10
  25. 2024ZigZagkv: Dynamic KV Cache Compression for Long-context Modeling based on Layer Uncertaintycited by 7
  26. 2024KVPruner: Structural Pruning for Faster and Memory-Efficient Large Language Modelscited by 6
  27. 2025Adaptive KV-Cache Compression without Manually Setting Budgetcited by 3

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026PAGE: Partition-Aware Gated KV-Cache Evictioncited by 0
  2. 2026Risk-Controlled KV-Cache Eviction: From Memory Budgets to Risk Targetscited by 0