mlsanityData to 14 Oct 2026

LeanK: Learnable K Cache Channel Pruning for Efficient Decoding

arXiv 2508.02215 ↗First posted 4 Aug 2025cs.LG cs.AI cs.CLcited by 7

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) enable long-context tasks but face efficiency challenges due to the growing key-value (KV) cache. We propose LeanK, a learning-based method that prunes unimportant key (K) cache channels by leveraging static channel sparsity. With a novel two-stage training process, LeanK learns channel-wise static mask that could satisfy specific sparsity ratio and hardware alignment requirement. LeanK reduces GPU memory and accelerates decoding without sacrificing accuracy. Experiments demonstrate up to 70% K cache and 16%-18% V cache memory reduction. Custom decoding kernel enables 1.3x speedup for attention computation. We also provide insights into model channels and attention heads during long-context inference by analyzing the learned importance distribution. Our code is available at https://aka.ms/LeanK.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 24 of its 31 references that are in the map. First posted on arXiv 4 Aug 2025.

Builds on

24 of 31 references.

References · 24

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  3. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  4. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  5. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,346
  6. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,121
  7. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 880
  8. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 637
  9. 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inferencecited by 501
  10. 2024DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Headscited by 288
  11. 2024Retrieval Head Mechanistically Explains Long-Context Factualitycited by 232
  12. 2024The Unreasonable Ineffectiveness of the Deeper Layerscited by 226
12 more
  1. 2023LooGLE: Can Long-Context Language Models Understand Long Contexts?cited by 202
  2. 2024A Survey on Recent Advances in LLM-Based Multi-turn Dialogue Systemscited by 170
  3. 2024Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Modelscited by 146
  4. 2023SparQ Attention: Bandwidth-Efficient LLM Inferencecited by 119
  5. 2024RetrievalAttention: Accelerating Long-Context LLM Inference via Vector Retrievalcited by 116
  6. 2024Round and Round We Go! What makes Rotary Positional Encodings useful?cited by 115
  7. 2024MagicPIG: LSH Sampling for Efficient LLM Generationcited by 111
  8. 2024ThinK: Thinner Key Cache by Query-Driven Pruningcited by 68
  9. 2024Post-Training Sparse Attention with Double Sparsitycited by 45
  10. 2025TileLang: A Composable Tiled Programming Model for AI Systemscited by 43
  11. 2025Quantization Meets Reasoning: Exploring LLM Low-Bit Quantization Degradation for Mathematical Reasoningcited by 37
  12. 2024On the token distance modeling ability of higher RoPE attention dimensioncited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 6.

7 papers.

  1. 2026RAP: KV-Cache Compression via RoPE-Aligned Pruningcited by 3
  2. 2025The Structural Scalpel: Automated Contiguous Layer Pruning for Large Language Modelscited by 2
  3. 2026HieraSparse: Hierarchical Semi-Structured Sparse KV Attentioncited by 0
  4. 2026Rotation-Aligned Key Channel Pruning for Efficient Vision-Language Model Inferencecited by 0
  5. 2026S4R: Selective Sampling, Subspaces, and Sparse Reconstruction for Compressed Long-Context KV Cachingcited by 0
  6. 2026WavePrune: One period is often enough for RoPEcited by 0
  7. 2026PN-Mix: Pair-Norm Guided Mixed-Precision Key Cache Quantization for Efficient Long-Output LLM Inferencecited by 0