mlsanityData to 14 Oct 2026

QCFuse: Query-Aware Cache Fusion via Compressed View for Efficient RAG Serving

arXiv 2606.05875 ↗First posted 4 Jun 2026cs.AI cs.DBcited by 3

On the map: Retrieval-augmented generation → Generation and context integration

Abstract (arXiv)

Retrieval-augmented generation (RAG) improves large language model (LLM) answer quality by grounding generation in external evidence, but processing retrieved contexts makes the prefill stage a dominant serving cost. RAG cache fusion reduces this cost by reusing precomputed key-value (KV) caches for retrieved chunks and selectively recomputing tokens under the current prompt. Existing selectors, however, face a dilemma between quality and efficiency: fast query-agnostic or final-layer query-to-context selectors can miss request-relevant evidence, whereas full-view query-aware selectors require broad context and layer visibility before recomputation and therefore stall the layer-wise cache-fusion pipeline. We present QCFuse, a compressed-view query-aware selector for RAG cache fusion. QCFuse uses chunk-anchor query probing to condition user-query states on compact per-chunk anchors and critical-layer profiling to identify recomputation tokens without all-layer inspection. We implement QCFuse in SGLang and evaluate it on four open-weight LLMs across six datasets. QCFuse reaches full-prefill-level quality. At matched quality, QCFuse achieves an average prefill-time speedup of 1.7x over full prefill and 1.5x over ProphetKV, the strongest quality-preserving baseline.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Generation and context integration, one of 15 directions in Retrieval-augmented generation. Builds on 46 of its 66 references that are in the map. First posted on arXiv 4 Jun 2026.

Builds on

46 of 66 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023RECOMP: Improving Retrieval-Augmented LMs with Compression and Selective Augmentation6 Oct
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks22 May
  3. 2020REALM: Retrieval-Augmented Language Model Pre-Training10 Feb

Other references · 43

  1. 2017Attention Is All You Needcited by 60,361
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  4. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  5. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,824
  6. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
  7. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  8. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  9. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  10. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  11. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,346
  12. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,346
31 more
  1. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,282
  2. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  3. 2018Self-Attention with Relative Position Representationscited by 1,120
  4. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,103
  5. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,059
  6. 2019Attention is not Explanationcited by 929
  7. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 916
  8. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 880
  9. 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 717
  10. 2024Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Servecited by 526
  11. 2024Quest: Query-Aware Sparsity for Efficient Long-Context LLM Inferencecited by 501
  12. 2023Model Tells You What to Discard: Adaptive KV Cache Compression for LLMscited by 474
  13. 2023LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compressioncited by 453
  14. 2023Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Timecited by 439
  15. 2024PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funnelingcited by 432
  16. 2025Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attentioncited by 407
  17. 2024MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attentioncited by 383
  18. 2016NewsQA: A Machine Comprehension Datasetcited by 331
  19. 2019Analyzing the Structure of Attention in a Transformer Language Modelcited by 243
  20. 2024Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inferencecited by 235
  21. 2024CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusioncited by 223
  22. 2024QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Servingcited by 208
  23. 2023CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Servingcited by 203
  24. 2024InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memorycited by 197
  25. 2024PyramidInfer: Pyramid KV Cache Compression for High-throughput LLM Inferencecited by 147
  26. 2025LMCache: An Efficient KV Cache Layer for Enterprise-Scale LLM Inferencecited by 126
  27. 2024PQCache: Product Quantization-based KVCache for Long Context LLM Inferencecited by 95
  28. 2025KVzip: Query-Agnostic KV Cache Compression with Context Reconstructioncited by 95
  29. 2025Expected Attention: KV Cache Compression by Estimating Attention from Future Queries Distributioncited by 74
  30. 2025Pneuma: Leveraging LLMs for Tabular Data Representation and Retrieval in an End-to-End Systemcited by 23
  31. 2025AlayaDB: The Data Foundation for Efficient and Effective Long-context LLM Inferencecited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026SemPIC: Learning Semantic Position-Independent KV Cachescited by 2
  2. 2026CONDUIT: A Unified Residual-Stream Restoration Framework for KV Cache Reuse in Vision-Language Modelscited by 0
  3. 2026Budgeted KV-Cache Repair for Evolving Contextscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026QCFuse: Query-Centric Cache Fusion for Efficient RAG Inference
  2. 2026From Prefix Cache to Fusion RAG Cache: Accelerating LLM Inference in Retrieval-Augmented Generation
  3. 2025Cache-Craft: Managing Chunk-Caches for Efficient Retrieval-Augmented Generation
  4. 2026Fine-Tuning a KV Cache Concatenation-Aware Model or Recomputing KV Caches? Why Not Both?
  5. 2026ProphetKV: User-Query-Driven Selective Recomputation for Efficient KV Cache Reuse in Retrieval-Augmented Generation
  6. 2024RazorAttention: Efficient KV Cache Compression Through Retrieval Heads
  7. 2025CacheClip: Accelerating RAG with Effective KV Cache Reuse
  8. 2024Toward Optimal Search and Retrieval for RAG