mlsanityData to 14 Oct 2026

Shared-Prefix KV Reuse Across Standard LoRA Adapters: Quality and Serving Tradeoffs

arXiv 2609.17109 ↗First posted 15 Sep 2026cs.AI cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

A common small-model deployment runs one shared backbone with several LoRA specialists that answer over the same context. Serving them naively re-prefills that shared context once per specialist. We study a narrow, practical question: for already-trained standard LoRA adapters -- not adapters retrained for cache compatibility -- how much task quality is preserved if the backbone's prefill KV cache is computed once and reused across specialists, and what does that buy in serving cost? On a Qwen3-1.7B backbone with two adapters (extractive QA on HotpotQA, arithmetic reasoning on GSM8K), we sweep the boundary at which the specialist takes over from the reused base cache and measure paired quality differences and serving cost. Full-prefix reuse had the lowest prefill cost and a small quality difference on held-out GSM8K (Delta = -4.6 EM at a 160-token budget; -3.0 at 320 tokens; -0.8 under a second training seed -- all favoring native, only the first excluding zero, and the magnitude not consistent). Partial recomputation provided no demonstrated advantage. Neither quality equivalence nor a general boundary-selection rule is established. We also report a closed-form ridge KV translator that did not beat direct reuse, and specialist-dependence contrasts whose intervals all include zero. The measured serving benefit is warm-cache time-to-first-token, which grows with context (~16x at 8K); two-branch peak memory was only 12% lower and, on inspection, the prefix was never physically shared across branches -- this implementation reuses KV values but copies their storage, so shared-cache memory savings are not achieved.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 10 of its 11 references that are in the map. First posted on arXiv 15 Sep 2026.

Builds on

10 of 11 references.

References · 10

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2025Qwen3 Technical Reportcited by 8,820
  4. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  5. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  6. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  7. 2026Cross-Model KV Cache Transfer in LLM Families: A Closed-Form Linear Mapping for Prefill Reusecited by 15
  8. 2025Activated LoRA: Fine-tuned LLMs for Intrinsicscited by 12
  9. 2025Efficient Multi-Adapter LLM Serving via Cross-Model KV-Cache Reuse with Activated LoRAcited by 8
  10. 2026CacheBridge: Efficient Cross-Model KV Cache Transfercited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Budgeted Cache Repair for Cross-Context KV-Cache Reuse
  2. 2026To Cache or Not to Cache: Understanding and Mitigating Safety Incompatibility in Cross-Model KV Cache Reuse
  3. 2026A Universal Context-Reuse Layer for Cross-Model KV Sharing
  4. 2026KVShareArena: KV-Cache Reuse Across Contexts and Model Checkpoints
  5. 2026Adaptive KV Cache Reuse for Fast Long-Context LLM Serving
  6. 2026CacheReforge: Bounded Recovery for Stale KV Caches under Evolving Adapters
  7. 2026CacheRoute: Planned Prefix-Affinity Routing for Large-Scale LLM Serving
  8. 2026Evaluating the accuracy of KV cache reuse techniques