mlsanityData to 7 Oct 2026

S4R: Selective Sampling, Subspaces, and Sparse Reconstruction for Compressed Long-Context KV Caching

arXiv 2608.00528 ↗First posted 1 Aug 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

The growth of context window lengths in Large Language Models (LLMs) significantly enhances their long-context capabilities but incurs prohibitive memory costs due to the Key-Value (KV) cache. Although low-rank compression of KV cache is a promising remedy, existing methods face a dilemma: offline approaches depend on external calibration data, whereas online approaches incur substantial compute for full-prompt decomposition and reconstruction. In this paper, we propose S4R, which builds low-rank subspaces from selectively sampled tokens and computes attention over a sparsely reconstructed KV representation. S4R uses prompt-aware initialization to build initial key/value bases from a representative prompt subset, trading off calibration-data dependence against prefilling cost. Because fully reconstructing the cache at every decoding step is prohibitively expensive and hurts throughput, we further adopt sparse reconstruction to retain only informative positions during decoding. Extensive experiments on LongBench and RULER with Llama and Qwen model families show that S4R achieves up to 5× KV compression with near full-cache accuracy, combining the efficiency of fixed compression with the adaptability of prompt-dependent methods.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 40 of its 54 references that are in the map. First posted on arXiv 1 Aug 2026.

Builds on

40 of 54 references.

References · 40

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  3. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  4. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  5. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  6. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,378
  7. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,311
  8. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,270
  9. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 848
  10. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 604
  11. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 571
  12. 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inferencecited by 487
28 more
  1. 2023Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Timecited by 428
  2. 2024PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funnelingcited by 420
  3. 2024LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitaskscited by 367
  4. 2024DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Headscited by 283
  5. 2024Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inferencecited by 230
  6. 2024PyramidInfer: Pyramid KV Cache Compression for High-throughput LLM Inferencecited by 148
  7. 2024ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inferencecited by 132
  8. 2024A Survey on Large Language Model Acceleration based on KV Cache Managementcited by 125
  9. 2024Model Tells You Where to Merge: Adaptive KV Cache Merging for LLMs on Long-Context Taskscited by 94
  10. 2025KVzip: Query-Agnostic KV Cache Compression with Context Reconstructioncited by 89
  11. 2024Loki: Low-rank Keys for Efficient Sparse Attentioncited by 87
  12. 2024Eigen Attention: Attention in Low-Rank Space for KV Cache Compressioncited by 71
  13. 2024Palu: Compressing KV-Cache with Low-Rank Projectioncited by 59
  14. 2024LoRC: Low-Rank Compression for LLMs KV Cache with a Progressive Compression Strategycited by 50
  15. 2024NACL: A General and Effective KV Cache Eviction Framework for LLMs at Inference Timecited by 43
  16. 2025RocketKV: Accelerating Long-Context LLM Inference via Two-Stage KV Cache Compressioncited by 41
  17. 2024Effectively Compress KV Heads for LLMcited by 30
  18. 2025ReCalKV: Low-Rank KV Cache Compression via Head Reordering and Offline Calibrationcited by 16
  19. 2025xKV: Cross-Layer KV-Cache Compression via Aligned Singular Vector Extractioncited by 13
  20. 2024KV-Compress: Paged KV-Cache Compression with Variable Compression Rates per Attention Headcited by 12
  21. 2026STAR-KV: Low-Rank KV Cache Compression via Soft Thresholding for Adaptive Rank Controlcited by 11
  22. 2025CommonKV: Compressing KV Cache with Cross-layer Parameter Sharingcited by 8
  23. 2025SALS: Sparse Attention in Latent Space for KV cache Compressioncited by 7
  24. 2025LeanK: Learnable K Cache Channel Pruning for Efficient Decodingcited by 5
  25. 2024CSKV: Training-Efficient Channel Shrinking for KV Cache in Long-Context Scenarioscited by 4
  26. 2025DynaKV: Enabling Accurate and Efficient Long-Sequence LLM Decoding on Smartphonescited by 4
  27. 2026GRKV: Global Regression for Training-Free KV Cache Compression in Long-Context LLMscited by 4
  28. 2025TaDA: Training-free recipe for Decoding with Adaptive KV Cache Compression and Mean-centeringcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026PuzzleKV: Page-Wise Low-Rank Decomposition for KV Cache Compression
  2. 2026DynaCalKV: Key-Value Cache Compression via Head Grouping and Adaptive Rank Allocation
  3. 2025OjaKV: Context-Aware Online Low-Rank KV Cache Compression
  4. 2026ResidualKV: Residual-Based KV Cache Compression for Efficient Long-Context Inference
  5. 2026A JoLT for the KV cache: Near-Lossless KV Cache Compression via Joint Rank-bit Allocation
  6. 2025KVCompose: Efficient Structured KV Cache Compression with Composite Tokens
  7. 2026KV2: A Self-Refining KV Cache
  8. 2024A Simple and Effective L2 Norm-Based Strategy for KV Cache Compression