# FreqDepthKV: Frequency-Guided Depth Sharing for Robust KV Cache Compression in Long-Context LLM Inference

arXiv: https://arxiv.org/abs/2607.06519 · HTML: https://mlsanity.com/paper/2607.06519/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving.
- Builds on 42 of its 60 references that are in the map.
- First posted on arXiv 7 Jul 2026.

## Placed in

- Efficient LLM inference and serving / [KV cache optimization](https://mlsanity.com/inference/kv-cache-optimization/)

## Builds on (most cited first, 30 of 42)

- [A Survey on LLM-as-a-Judge](https://mlsanity.com/paper/2411.15594/) (2024)
- [Mooncake: A KVCache-centric Disaggregated Architecture for LLM Serving](https://mlsanity.com/paper/2407.00079/) (2024)
- [A Survey on Large Language Model Acceleration based on KV Cache Management](https://mlsanity.com/paper/2412.19442/) (2024)
- [MiniCache: KV Cache Compression in Depth Dimension for Large Language Models](https://mlsanity.com/paper/2405.14366/) (2024)
- [Hymba: A Hybrid-head Architecture for Small Language Models](https://mlsanity.com/paper/2411.13676/) (2024)
- [Model Tells You Where to Merge: Adaptive KV Cache Merging for LLMs on Long-Context Tasks](https://mlsanity.com/paper/2407.08454/) (2024)
- [Eigen Attention: Attention in Low-Rank Space for KV Cache Compression](https://mlsanity.com/paper/2408.05646/) (2024)
- [The Modern Mathematics of Deep Learning](https://mlsanity.com/paper/2105.04026/) (2021)
- [ChunkKV: Semantic-Preserving KV Cache Compression for Efficient Long-Context LLM Inference](https://mlsanity.com/paper/2502.00299/) (2025)
- [R-KV: Redundancy-aware KV Cache Compression for Reasoning Models](https://mlsanity.com/paper/2505.24133/) (2025)
- [KVSharer: Efficient Inference via Layer-Wise Dissimilar KV Cache Sharing](https://mlsanity.com/paper/2410.18517/) (2024)
- [Deep learning observables in computational fluid dynamics](https://mlsanity.com/paper/1903.03040/) (2019)
- [Attention Is All You Need for KV Cache in Diffusion LLMs](https://mlsanity.com/paper/2510.14973/) (2025)
- [Learn to Accumulate Evidence from All Training Samples: Theory and Practice](https://mlsanity.com/paper/2306.11113/) (2023)
- [SwiftKV: Fast Prefill-Optimized Inference with Knowledge-Preserving Model Transformation](https://mlsanity.com/paper/2410.03960/) (2024)
- [The Rise of Small Language Models in Healthcare: A Comprehensive Survey](https://mlsanity.com/paper/2504.17119/) (2025)
- [The Pitfalls of KV Cache Compression](https://mlsanity.com/paper/2510.00231/) (2025)
- [FastKV: Decoupling of Context Reduction and KV Cache Compression for Prefill-Decoding Acceleration](https://mlsanity.com/paper/2502.01068/) (2025)
- [Efficient Many-Shot In-Context Learning with Dynamic Block-Sparse Attention](https://mlsanity.com/paper/2503.08640/) (2025)
- [FAEDKV: Infinite-Window Fourier Transform for Unbiased KV Cache Compression](https://mlsanity.com/paper/2507.20030/) (2025)
- [Make Each Token Count: Towards Improving Long-Context Performance with KV Cache Eviction](https://mlsanity.com/paper/2605.09649/) (2026)
- [Compressing KV Cache for Long-Context LLM Inference with Inter-Layer Attention Similarity](https://mlsanity.com/paper/2412.02252/) (2024)
- [TailorKV: A Hybrid Framework for Long-Context Inference via Tailored KV Cache Optimization](https://mlsanity.com/paper/2505.19586/) (2025)
- [EvolKV: Evolutionary KV Cache Compression for LLM Inference](https://mlsanity.com/paper/2509.08315/) (2025)
- [SpindleKV: A Novel KV Cache Reduction Method Balancing Both Shallow and Deep Layers](https://mlsanity.com/paper/2507.06517/) (2025)
- [AI Agents as Universal Task Solvers](https://mlsanity.com/paper/2510.12066/) (2025)
- [Q-KVComm: Efficient Multi-Agent Communication Via Adaptive KV Cache Compression](https://mlsanity.com/paper/2512.17914/) (2025)
- [ReST-KV: Robust KV Cache Eviction with Layer-wise Output Reconstruction and Spatial-Temporal Smoothing](https://mlsanity.com/paper/2605.08840/) (2026)
- [Lossless KV Cache Compression to 2%](https://mlsanity.com/paper/2410.15252/) (2024)
- [Training Transformers for KV Cache Compressibility](https://mlsanity.com/paper/2605.05971/) (2026)

## Abstract (arXiv)

Long-context LLM inference is increasingly limited by the memory and bandwidth cost of KV caches, yet aggressive compression can remove the layer-specific evidence needed for retrieval and multi-step reasoning. We introduce FreqDepthKV, an inference-time cache compression method that factorizes adjacent-layer KV states into shared low-frequency depth components and sparse high-frequency residuals. A lightweight online probe assigns attention heads to shared-depth, residual-depth, or exact cache modes according to their contribution to reconstruction-sensitive attention logits, allowing the compression policy to adapt to prompt structure without retraining. Across long-context question answering, needle retrieval, summarization, and code generation benchmarks, FreqDepthKV preserves task accuracy under substantially smaller cache budgets. With a 32k-token prefill window, FreqDepthKV reaches 58.3 Exact Match, 63.0 F1, 32.5 ROUGE-L, and 48.1 pass@1, closely matching full KV while outperforming prior compressed-cache methods. It also improves decoding throughput to 70.4 tokens/s, reduces TTFT to 2.06 seconds, and lowers peak KV memory to 6.2 GB, achieving a 3.9x effective compression ratio.