mlsanityData to 14 Oct 2026

KV Admission: Learning What to Write for Efficient Long-Context LLM Inference

arXiv 2512.17452 ↗First posted 19 Dec 2025cs.LG cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

Long-context LLM inference is bottlenecked by the quadratic attention complexity and linear Key-Value (KV) cache growth. Prior approaches mitigate this via post-hoc selection or eviction but overlook the root inefficiency: indiscriminate token admission. In this paper, we formalize KV management as a causal system of three primitives: KV Admission, Selection, and Eviction. We instantiate KV Admission via Write-Gated KV (WG-KV), a lightweight mechanism that learns to predict token utility before cache entry. By filtering out redundant states early to maintain a compact global cache alongside a sliding local cache, WG-KV significantly reduces memory usage and accelerates both prefill and decode phases. Our results demonstrate that learning what to write is a principled and practical recipe for efficient long-context inference. Code is available at https://github.com/EMCLab-Sinica/WG-KV.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 92 of its 125 references that are in the map. First posted on arXiv 19 Dec 2025.

Builds on

92 of 125 references.

References · 92

  1. 2017Attention Is All You Needcited by 60,361
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  3. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  4. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  5. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  6. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  7. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,739
  8. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  9. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,605
  10. 2019Root Mean Square Layer Normalizationcited by 1,480
  11. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
  12. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,160
38 more
  1. 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,148
  2. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  3. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,128
  4. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 880
  5. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 832
  6. 2019Compressive Transformers for Long-Range Sequence Modellingcited by 726
  7. 2017The NarrativeQA Reading Comprehension Challengecited by 597
  8. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 591
  9. 2017Learning Sparse Neural Networks through L0 Regularizationcited by 583
  10. 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inferencecited by 501
  11. 2023Model Tells You What to Discard: Adaptive KV Cache Compression for LLMscited by 474
  12. 2023Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Timecited by 439
  13. 2020Efficient Intent Detection with Dual Sentence Encoderscited by 433
  14. 2025Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attentioncited by 407
  15. 2019An Evaluation Dataset for Intent Classification and Out-of-Scope Predictioncited by 386
  16. 2024MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attentioncited by 383
  17. 2024LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitaskscited by 379
  18. 2024∞Bench: Extending Long Context Evaluation Beyond 100K Tokenscited by 367
  19. 2023SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verificationcited by 365
  20. 2022The Stack: 3 TB of permissively licensed source codecited by 359
  21. 2024DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Headscited by 288
  22. 2024InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Managementcited by 245
  23. 2024Retrieval Head Mechanistically Explains Long-Context Factualitycited by 232
  24. 2025MoBA: Mixture of Block Attention for Long-Context LLMscited by 198
  25. 2025Plan-and-Act: Improving Planning of Agents for Long-Horizon Taskscited by 195
  26. 2025MultiChallenge: A Realistic Multi-Turn Conversation Evaluation Benchmark Challenging to Frontier LLMscited by 180
  27. 2019Benchmarking Natural Language Understanding Services for building Conversational Agentscited by 166
  28. 2024PyramidInfer: Pyramid KV Cache Compression for High-throughput LLM Inferencecited by 147
  29. 2025XAttention: Block Sparse Attention with Antidiagonal Scoringcited by 137
  30. 2024ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inferencecited by 134
  31. 2024Reducing Transformer Key-Value Cache Size with Cross-Layer Attentioncited by 131
  32. 2024MiniCache: KV Cache Compression in Depth Dimension for Large Language Modelscited by 125
  33. 2024SeerAttention: Learning Intrinsic Sparse Attention in Your LLMscited by 124
  34. 2025FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inferencecited by 123
  35. 2024Transformers are Multi-State RNNscited by 120
  36. 2025SpargeAttention: Accurate and Training-free Sparse Attention Accelerating Any Model Inferencecited by 120
  37. 2023SparQ Attention: Bandwidth-Efficient LLM Inferencecited by 119
  38. 2024Dynamic Memory Compression: Retrofitting LLMs for Accelerated Inferencecited by 116

50 most cited of 92 All 92 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026SPHERICAL KV: Angle-Domain Attention and Rate-Distortion Retention for Efficient Long-Context Inferencecited by 2
  2. 2026Shallow Prefill, Deep Decoding: Efficient Long-Context Inference via Layer-Asymmetric KV Visibilitycited by 1
  3. 2026Keyless Attention: Value-Space Routing and Value-Only Caching for Efficient Transformerscited by 0
  4. 2026ReSS: Residual-Restoring Sparse Attention for 3D Vision Transformerscited by 0