mlsanityData to 7 Oct 2026

EchoPress: Query-Agnostic KV Cache Pruning via Virtual Context Reconstruction

arXiv 2610.00412 ↗First posted 30 Sep 2026cs.LGcited by 1

On the map: Efficient LLM inference and serving → KV cache optimization

Abstract (arXiv)

KV cache pruning reduces long-context inference memory usage by evicting less important key-value pairs. KVzip estimates importance through context reconstruction: prompting a model to repeat the context chunk by chunk. This achieves strong compression quality at the cost of additional forward passes. Learned approximations reduce this cost but require model-specific training. We analyze how KVzip identifies important cached information and show how to approximate its reconstruction scores using information already computed during prefill. These findings motivate EchoPress, a training-free method that approximates reconstruction attention using queries and keys from standard prefill. For each request, it reconstructs only the first chunk to calibrate importance scores for the remaining context. Experiments on LongBench and RULER with Qwen3-8B and Llama-3.1-8B-Instruct show that EchoPress matches KVzip in task accuracy across eviction ratios from 50% to 90%, while reducing compression overhead by a factor of 1.7-19.6 and total prefill time by a factor of up to 2.9. Code is available at https://github.com/ljwljwljwljw/kvpress/tree/echo-press.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 58 of its 60 references that are in the map. First posted on arXiv 30 Sep 2026.

Builds on

58 of 60 references.

Closest ancestors · 30

Same direction, newest first

  1. 2026RestoreKV: Recovering Full-Cache Behavior Under Aggressive Query-Agnostic KV Cache Eviction2 Aug
  2. 2026Predicting Future Utility: Global Combinatorial Optimization for Task-Agnostic KV Cache Eviction9 Feb
  3. 2026Fast KVzip: Efficient and Accurate LLM Inference with Gated KV Eviction25 Jan
  4. 2026KVzap: Fast, Adaptive, and Faithful KV Cache Pruning12 Jan
  5. 2025Cache What Lasts: Token Retention for Memory-Bounded KV Cache in LLMs3 Dec
  6. 2025Expected Attention: KV Cache Compression by Estimating Attention from Future Queries Distribution1 Oct
  7. 2025EpiCache: Episodic KV Cache Management for Long-Term Conversation on Resource-Constrained Environments22 Sep
  8. 2025Compactor: Calibrated Query-Agnostic KV Cache Compression with Approximate Leverage Scores10 Jul
  9. 2025Inference-Time Hyper-Scaling with KV Cache Compression5 Jun
  10. 2025KVzip: Query-Agnostic KV Cache Compression with Context Reconstruction29 May
  11. 2025KeyDiff: Key Similarity-Based KV Cache Eviction for Long-Context LLM Inference in Resource-Constrained Environments21 Apr
  12. 2025Q-Filters: Leveraging QK Geometry for Efficient KV Cache Compression4 Mar
18 more
  1. 2024SCBench: A KV Cache-Centric Analysis of Long-Context Methods13 Dec
  2. 2024DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads14 Oct
  3. 2024Locret: Enhancing Eviction in Long-Context LLM Inference with Trained Retaining Heads on Consumer-Grade Devices2 Oct
  4. 2024NACL: A General and Effective KV Cache Eviction Framework for LLMs at Inference Time7 Aug
  5. 2024ThinK: Thinner Key Cache by Query-Driven Pruning30 Jul
  6. 2024Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inference16 Jul
  7. 2024InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Management28 Jun
  8. 2024A Simple and Effective L2 Norm-Based Strategy for KV Cache Compression17 Jun
  9. 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inference16 Jun
  10. 2024PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling4 Jun
  11. 2024CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusion26 May
  12. 2024PyramidInfer: Pyramid KV Cache Compression for High-throughput LLM Inference21 May
  13. 2024SnapKV: LLM Knows What You are Looking for Before Generation22 Apr
  14. 2024No Token Left Behind: Reliable KV Cache Compression via Importance-Aware Mixed Precision Quantization28 Feb
  15. 2023Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs3 Oct
  16. 2023Efficient Memory Management for Large Language Model Serving with PagedAttention12 Sep
  17. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models24 Jun
  18. 2023Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Time26 May

Other references · 28

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2025Qwen3 Technical Reportcited by 8,195
  3. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  4. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  5. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  6. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,378
  7. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,311
  8. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,270
  9. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,107
  10. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 604
  11. 2023YaRN: Efficient Context Window Extension of Large Language Modelscited by 584
  12. 2024MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attentioncited by 375
16 more
  1. 2023Prompt Cache: Modular Attention Reuse for Low-Latency Inferencecited by 255
  2. 2023Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Timecited by 247
  3. 2025MoBA: Mixture of Block Attention for Long-Context LLMscited by 188
  4. 2024SeerAttention: Learning Intrinsic Sparse Attention in Your LLMscited by 121
  5. 2024Transformers are Multi-State RNNscited by 120
  6. 2024RetrievalAttention: Accelerating Long-Context LLM Inference via Vector Retrievalcited by 114
  7. 2024MagicPIG: LSH Sampling for Efficient LLM Generationcited by 112
  8. 2024ZipCache: Accurate and Efficient KV Cache Quantization with Salient Token Identificationcited by 86
  9. 2023Dynamic Context Pruning for Efficient and Interpretable Autoregressive Transformerscited by 67
  10. 2025R-KV: Redundancy-aware KV Cache Compression for Reasoning Modelscited by 52
  11. 2024EPIC: Efficient Position-Independent Caching for Serving Large Language Modelscited by 45
  12. 2023Compressed Context Memory For Online Language Model Interactioncited by 42
  13. 2024InfiniPot: Infinite Context Processing on Memory-Constrained LLMscited by 27
  14. 2025LLM-Microscope: Uncovering the Hidden Role of Punctuation in Context Memory of Transformerscited by 12
  15. 2025Which Heads Matter for Reasoning? RL-Guided KV Cache Compressioncited by 10
  16. 2025Beyond RAG: Task-Aware KV Cache Compression for Comprehensive Knowledge Reasoningcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Rehearse Everything, Remember Nothing: Attic-KV Rehearses What Will Be Readcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026KV2: A Self-Refining KV Cache
  2. 2026TaskPress: Query-Agnostic KV Cache Compression via Task-Guided Pruning
  3. 2026Behavior-Preserving KV Cache Compression
  4. 2026KVFetch: Temporal Prefetching for the Missing Half of KV Cache Compression
  5. 2026Where Matters More Than What: Decoding-aligned KV Cache Compression via Position-aware Pseudo Queries
  6. 2024KV-Compress: Paged KV-Cache Compression with Variable Compression Rates per Attention Head
  7. 2026EchoKV: Efficient KV Cache Compression via Similarity-Based Reconstruction
  8. 2026Make Each Token Count: Towards Improving Long-Context Performance with KV Cache Eviction