mlsanityData to 7 Oct 2026

RelaxKV: Recomputation Guided by the Query with Sparse Context Attention for Efficient KV Cache Reuse

arXiv 2609.33503 ↗First posted 27 Sep 2026cs.AIcited by 0

On the map: Retrieval-augmented generation → Generation and context integration

also in 1 more
Abstract (arXiv)

Cross-request KV caching reduces the prefill cost of Retrieval-Augmented Generation (RAG), but conventional prefix caching severely limits cache reuse across requests. Position-Independent Caching (PIC) removes this constraint by reusing independent chunks, but their KV states miss cross-chunk interactions. Existing methods selectively recompute token states to recover these missing interactions, but primarily allocate the recomputation budget to selecting which states to recompute, while fixing the recomputation context to the full causal prefix. We introduce RelaxKV, which formulates selective cache repair as a joint allocation problem over repair targets and recomputation context. Guided by the user query, RelaxKV identifies layer-specific repair targets and restricts their recomputation to a query-relevant context, reducing attention computation. Across four decoder models, RelaxKV at a 15% anchor ratio improves aggregate LongBench performance over ProphetKV on all models. On Qwen3-14B, RelaxKV provides a stronger quality-TTFT trade-off than ProphetKV across a 5%-30% anchor-ratio sweep, and achieves the best selective results on RULER-MV and LV-Eval at 16K and 32K context lengths. Controlled ablations further demonstrate the importance of recomputation context selection.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Generation and context integration, one of 15 directions in Retrieval-augmented generation. Builds on 23 of its 23 references that are in the map. First posted on arXiv 27 Sep 2026.

Builds on

23 of 23 references.

Closest ancestors · 3

Same direction, newest first

  1. 2026ProphetKV: User-Query-Driven Selective Recomputation for Efficient KV Cache Reuse in Retrieval-Augmented Generation31 Jan
  2. 2025Cache-Craft: Managing Chunk-Caches for Efficient Retrieval-Augmented Generation5 Feb
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks22 May

Other references · 20

  1. 2017Attention Is All You Needcited by 59,932
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2025Qwen3 Technical Reportcited by 8,195
  4. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  5. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  6. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,378
  7. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,270
  8. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 893
  9. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 848
  10. 2024Phi-4 Technical Reportcited by 675
  11. 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inferencecited by 487
  12. 2024CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusioncited by 211
8 more
  1. 2025KVzip: Query-Agnostic KV Cache Compression with Context Reconstructioncited by 89
  2. 2024LV-Eval: A Balanced Long-Context Benchmark with 5 Length Levels Up to 256Kcited by 80
  3. 2025KVLink: Accelerating Large Language Models via Efficient KV Cache Reusecited by 60
  4. 2024EPIC: Efficient Position-Independent Caching for Serving Large Language Modelscited by 45
  5. 2025KVShare: An LLM Service System with Efficient and Effective Multi-Tenant KV Cache Reusecited by 22
  6. 2026KV Packet: Recomputation-Free Context-Independent KV Caching for LLMscited by 10
  7. 2025Compress, Gather, and Recompute: REFORMing Long-Context Processing in Transformerscited by 3
  8. 2026QUOKA: Query-Oriented KV Selection For Efficient LLM Prefillcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026RestoreKV: Recovering Full-Cache Behavior Under Aggressive Query-Agnostic KV Cache Eviction
  2. 2026ATTUNER: Recomputation-Free KV Cache Reuse via Query-Side Adaptation
  3. 2026KVBoost: Chunk-Level Key-Value Cache Reuse with Deviation-Guided Recomputation for Efficient Large Language Model Inference
  4. 2026ResKV: Reconstructing Omitted Attention Contributions for Fixed-Budget KV Cache Compression
  5. 2026CacheRepair: Learning to Repair Cross-Chunk Context in RAG for KV Cache Fusion
  6. 2026KV2: A Self-Refining KV Cache
  7. 2026Request Order Matters: Cache-History Sensitivity in Selective KV-Cache Reuse for Rolling Agents
  8. 2026Fine-Tuning a KV Cache Concatenation-Aware Model or Recomputing KV Caches? Why Not Both?