mlsanityData to 14 Oct 2026

FreqDepthKV: Frequency-Guided Depth Sharing for Robust KV Cache Compression in Long-Context LLM Inference

arXiv 2607.06519 ↗First posted 7 Jul 2026cs.AIcited by 0

On the map: Efficient LLM inference and serving → KV cache optimization

Abstract (arXiv)

Long-context LLM inference is increasingly limited by the memory and bandwidth cost of KV caches, yet aggressive compression can remove the layer-specific evidence needed for retrieval and multi-step reasoning. We introduce FreqDepthKV, an inference-time cache compression method that factorizes adjacent-layer KV states into shared low-frequency depth components and sparse high-frequency residuals. A lightweight online probe assigns attention heads to shared-depth, residual-depth, or exact cache modes according to their contribution to reconstruction-sensitive attention logits, allowing the compression policy to adapt to prompt structure without retraining. Across long-context question answering, needle retrieval, summarization, and code generation benchmarks, FreqDepthKV preserves task accuracy under substantially smaller cache budgets. With a 32k-token prefill window, FreqDepthKV reaches 58.3 Exact Match, 63.0 F1, 32.5 ROUGE-L, and 48.1 pass@1, closely matching full KV while outperforming prior compressed-cache methods. It also improves decoding throughput to 70.4 tokens/s, reduces TTFT to 2.06 seconds, and lowers peak KV memory to 6.2 GB, achieving a 3.9x effective compression ratio.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 42 of its 60 references that are in the map. First posted on arXiv 7 Jul 2026.

Builds on

42 of 60 references.

Closest ancestors · 25

Same direction, newest first

  1. 2026The risk of KV cache compression1 Jul
  2. 2026Make Each Token Count: Towards Improving Long-Context Performance with KV Cache Eviction10 May
  3. 2026ReST-KV: Robust KV Cache Eviction with Layer-wise Output Reconstruction and Spatial-Temporal Smoothing9 May
  4. 2026Training Transformers for KV Cache Compressibility7 May
  5. 2026Sub-Token Routing for KV Cache Compression23 Apr
  6. 2026Knowledge Packs: Zero-Token Knowledge Delivery via KV Cache Injection22 Mar
  7. 2025KV Pareto: Systems-Level Optimization of KV Cache and Model Compression for Long Context Inference1 Dec
  8. 2025The Pitfalls of KV Cache Compression30 Sep
  9. 2025EvolKV: Evolutionary KV Cache Compression for LLM Inference10 Sep
  10. 2025Enhancing Large Multimodal Models with Adaptive Sparsity and KV Cache Compression28 Jul
  11. 2025FAEDKV: Infinite-Window Fourier Transform for Unbiased KV Cache Compression26 Jul
  12. 2025HCAttention: Extreme KV Cache Compression via Heterogeneous Attention Computing for LLMs26 Jul
13 more
  1. 2025SpindleKV: A Novel KV Cache Reduction Method Balancing Both Shallow and Deep Layers9 Jul
  2. 2025TailorKV: A Hybrid Framework for Long-Context Inference via Tailored KV Cache Optimization26 May
  3. 2025Key, Value, Compress: A Systematic Exploration of KV Cache Compression Techniques14 Mar
  4. 2025FastKV: Decoupling of Context Reduction and KV Cache Compression for Prefill-Decoding Acceleration3 Feb
  5. 2025ChunkKV: Semantic-Preserving KV Cache Compression for Efficient Long-Context LLM Inference1 Feb
  6. 2024A Survey on Large Language Model Acceleration based on KV Cache Management27 Dec
  7. 2024Compressing KV Cache for Long-Context LLM Inference with Inter-Layer Attention Similarity3 Dec
  8. 2024KVSharer: Efficient Inference via Layer-Wise Dissimilar KV Cache Sharing24 Oct
  9. 2024Lossless KV Cache Compression to 2%20 Oct
  10. 2024SwiftKV: Fast Prefill-Optimized Inference with Knowledge-Preserving Model Transformation4 Oct
  11. 2024Eigen Attention: Attention in Low-Rank Space for KV Cache Compression10 Aug
  12. 2024Model Tells You Where to Merge: Adaptive KV Cache Merging for LLMs on Long-Context Tasks11 Jul
  13. 2024MiniCache: KV Cache Compression in Depth Dimension for Large Language Models23 May

Other references · 17

  1. 2024A Survey on LLM-as-a-Judgecited by 1,423
  2. 2024Mooncake: A KVCache-centric Disaggregated Architecture for LLM Servingcited by 196
  3. 2024Hymba: A Hybrid-head Architecture for Small Language Modelscited by 99
  4. 2021The Modern Mathematics of Deep Learningcited by 63
  5. 2025R-KV: Redundancy-aware KV Cache Compression for Reasoning Modelscited by 54
  6. 2019Deep learning observables in computational fluid dynamicscited by 38
  7. 2025Attention Is All You Need for KV Cache in Diffusion LLMscited by 32
  8. 2023Learn to Accumulate Evidence from All Training Samples: Theory and Practicecited by 29
  9. 2025The Rise of Small Language Models in Healthcare: A Comprehensive Surveycited by 18
  10. 2025Efficient Many-Shot In-Context Learning with Dynamic Block-Sparse Attentioncited by 12
  11. 2025AI Agents as Universal Task Solverscited by 9
  12. 2025Q-KVComm: Efficient Multi-Agent Communication Via Adaptive KV Cache Compressioncited by 9
5 more
  1. 2025Compute-in-Memory Implementation of State Space Models for Event Sequence Processingcited by 6
  2. 2017A multitask deep learning model for real-time deployment in embedded systemscited by 3
  3. 2026Monodense Deep Neural Model for Determining Item Price Elasticitycited by 3
  4. 2026PolyKV: A Shared Asymmetrically-Compressed KV Cache Pool for Multi-Agent LLM Inferencecited by 3
  5. 2025Building Large-Scale English-Romanian Literary Translation Resources with Open Modelscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026CompressKV: Semantic-Retrieval-Guided KV-Cache Compression for Resource-Efficient Long-Context LLM Inference
  2. 2026Distance-KV: Exploiting Relative Distance for Efficient Long-Context Inference
  3. 2025RocketKV: Accelerating Long-Context LLM Inference via Two-Stage KV Cache Compression
  4. 2026ResidualKV: Residual-Based KV Cache Compression for Efficient Long-Context Inference
  5. 2025KVCompose: Efficient Structured KV Cache Compression with Composite Tokens
  6. 2026HeteroCache: A Dynamic Retrieval Approach to Heterogeneous KV Cache Compression for Long-Context LLM Inference
  7. 2026C2KV: Compressed and Composable KV Cache Reuse for Efficient LLM Inference
  8. 2026KVSculpt: KV Cache Compression as Distillation