KVReviver: Reversible KV Cache Compression with Sketch-Based Token Reconstruction
On the map: Efficient LLM inference and serving → KV cache optimization
Abstract (arXiv)
As the context length of current large language models (LLMs) rapidly increases, the memory demand for the Key-Value (KV) cache is becoming a bottleneck for LLM deployment and batch processing. Traditional KV cache compression methods typically involve permanently evicting or irreversibly merging "less important" tokens with low attention scores. This approach results in the unrecoverable loss of token information, which we call Contextual Amnesia, significantly degrading the model's information retrieval capability. To address this issue, we propose KVReviver, a reversible KV cache compression method based on the sketch algorithm. This method allows reconstructing compressed tokens from an additional data structure, thus enabling full-scale computation within limited memory. Experiments showed that in 2k-length contexts, it requires only 10% of KV Cache budget while maintaining identical end-to-end inference accuracy. For 32k-length contexts, it achieves equivalent or comparable accuracy ~2% accuracy loss) using merely 25% of KV Cache budget.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 18 of its 30 references that are in the map. First posted on arXiv 1 Dec 2025.
Builds on
18 of 30 references.
Closest ancestors · 7
Same direction, newest first
- 2024DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads14 Oct
- 2024Palu: Compressing KV-Cache with Low-Rank Projection30 Jul
- 2024PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling4 Jun
- 2024Reducing Transformer Key-Value Cache Size with Cross-Layer Attention21 May
- 2024PyramidInfer: Pyramid KV Cache Compression for High-throughput LLM Inference21 May
- 2024SnapKV: LLM Knows What You are Looking for Before Generation22 Apr
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttention12 Sep
Other references · 11
- 2017Attention Is All You Needcited by 60,361
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
- 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
- 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
- 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,317
- 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,135
- 2016Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyondcited by 812
- 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 591
- 2021Group Fisher Pruning for Practical Network Compressioncited by 131
- 2024Dynamic Memory Compression: Retrofitting LLMs for Accelerated Inferencecited by 116
- 2024MagicPIG: LSH Sampling for Efficient LLM Generationcited by 111
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025Compactor: Calibrated Query-Agnostic KV Cache Compression with Approximate Leverage Scores
- 2024KV-Compress: Paged KV-Cache Compression with Variable Compression Rates per Attention Head
- 2024More Tokens, Lower Precision: Towards the Optimal Token-Precision Trade-off in KV Cache Compression
- 2025KVCompose: Efficient Structured KV Cache Compression with Composite Tokens
- 2025LagKV: Lag-Relative Information of the KV Cache Tells Which Tokens Are Important
- 2024Compressing KV Cache for Long-Context LLM Inference with Inter-Layer Attention Similarity
- 2024Get More with LESS: Synthesizing Recurrence with KV Cache Compression for Efficient LLM Inference
- 2026DepthWeave-KV: Token-Adaptive Cross-Layer Residual Factorization for Long-Context KV Cache Compression