mlsanityData to 14 Oct 2026

KV-Repurpose: A Testbed for Evaluating Non-Prefix KV-Cache Reuse

arXiv 2610.13060 ↗First posted 9 Oct 2026cs.CLcited by 0

On the map: Efficient LLM inference and serving → KV cache optimization

Abstract (arXiv)

KV-cache reuse in large language model serving is commonly limited to exact prompt prefixes, preventing reuse when the same retrieved document appears after different system prompts, conversation histories, or dynamic metadata. We present KV-Repurpose, an open-source, vLLM-based system for developing and evaluating segment-wise KV-cache reuse beyond exact prefixes. The system combines cache-aware segment scheduling, prompt construction, capacity management, workload generation, and runtime monitoring. Experiments on long-context benchmarks show that naive segment reuse largely preserves output quality for single-document question answering and summarization, while exposing clear limitations in cross-document reasoning. Under a Zipfian RAG-style workload in which 20% of requests share a fixed prefix, KV-Repurpose achieves a substantially higher token-level cache hit rate than prefix caching at the same 30 GB cache capacity, 63.9% versus 3.3%, resulting in lower TTFT. Even with 1 TB of cache capacity, prefix caching reaches only a 17.9% hit rate in this workload. Source code is available under the Apache-2.0 license at https://github.com/taiga333/eval-kv-repurpose.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 23 of its 33 references that are in the map. First posted on arXiv 9 Oct 2026.

Builds on

23 of 33 references.

Closest ancestors · 9

Same direction, newest first

  1. 2026C2KV: Compressed and Composable KV Cache Reuse for Efficient LLM Inference20 Jul
  2. 2026RedKnot: Efficient Long-Context LLM Serving with Head-Aware KV Reuse and SegPagedAttention4 Jun
  3. 2025MEPIC: Memory Efficient Position Independent Caching for LLM Serving18 Dec
  4. 2025LMCache: An Efficient KV Cache Layer for Enterprise-Scale LLM Inference8 Oct
  5. 2025KVLink: Accelerating Large Language Models via Efficient KV Cache Reuse21 Feb
  6. 2024TurboRAG: Accelerating Retrieval-Augmented Generation with Precomputed KV Caches for Chunked Text10 Oct
  7. 2024CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusion26 May
  8. 2024Cost-Efficient Large Language Model Serving for Multi-turn Conversations with CachedAttention23 Mar
  9. 2023Efficient Memory Management for Large Language Model Serving with PagedAttention12 Sep

Other references · 14

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  4. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
  5. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,386
  6. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,282
  7. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  8. 2021A Dataset of Information-Seeking Questions and Answers Anchored in Research Paperscited by 486
  9. 2019Multi-News: a Large-Scale Multi-Document Summarization Dataset and Abstractive Hierarchical Modelcited by 431
  10. 2021QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarizationcited by 365
  11. 2021Efficient Attentions for Long Document Summarizationcited by 345
  12. 2023Prompt Cache: Modular Attention Reuse for Low-Latency Inferencecited by 268
2 more
  1. 2026LongBench Pro: A More Realistic and Comprehensive Bilingual Long-Context Evaluation Benchmarkcited by 19
  2. 2026Revolutionizing Long-Term Memory in AI: New Horizons with High-Capacity and High-Speed Storagecited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026KVShareArena: KV-Cache Reuse Across Contexts and Model Checkpoints
  2. 2026KV Packet: Recomputation-Free Context-Independent KV Caching for LLMs
  3. 2025KVShare: An LLM Service System with Efficient and Effective Multi-Tenant KV Cache Reuse
  4. 2026Request Order Matters: Cache-History Sensitivity in Selective KV-Cache Reuse for Rolling Agents
  5. 2025A3: Attention-Aware Accurate KV Cache Fusion for Fast Large Language Model Serving
  6. 2026An experimental study of KV cache reuse strategies in chunk-level caching systems
  7. 2026Evaluating the accuracy of KV cache reuse techniques
  8. 2026PCR: A Prefetch-Enhanced Cache Reuse System for Low-Latency RAG Serving