mlsanityData to 14 Oct 2026

QKVShare: Quantized KV-Cache Handoff for Multi-Agent On-Device LLMs

arXiv 2605.03884 ↗First posted 5 May 2026cs.AI cs.MAcited by 1

On the map: LLM agents and tool use → Software engineering coding agents

also in 1 more
Abstract (arXiv)

Multi-agent LLM systems on edge devices need to hand off latent context efficiently, but the practical choices today are expensive re-prefill or full-precision KV transfer. We study QKVShare, a framework for quantized KV-cache handoff between agents that combines token-level mixed-precision allocation, a self-contained CacheCard representation, and a HuggingFace-compatible cache injection path. Our current results support a narrower but clearer story than the original draft: on 150 GSM8K problems with Llama-3.1-8B-Instruct, adaptive quantization remains competitive under repeated handoff and shows its clearest gains against uniform quantization in deeper-hop, higher budget settings; for handoff latency, the QKVShare path reduces TTFT relative to full re prefill at every tested context, from 130.7 ms vs. 150.2 ms at nominal 1K context to 397.1 ms vs. 1029.7 ms at nominal 8K context;. Stage timing shows that post-injection generation, not card creation, dominates the current QKVShare latency path. These results position quantized KV handoff as a promising on-device systems direction while also highlighting the need for stronger controller ablations and apples-to-apples runtime comparisons.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 11 of its 15 references that are in the map. First posted on arXiv 5 May 2026.

Builds on

11 of 15 references.

Closest ancestors · 3

Same direction, newest first

  1. 2026LRAgent: Efficient KV Cache Sharing for Multi-LoRA LLM Agents1 Feb
  2. 2025Continuum: Efficient and Robust Multi-Turn LLM Agent Scheduling with KV Cache Time-to-Live4 Nov
  3. 2024DroidSpeak: KV Cache Sharing for Cross-LLM Communication and Multi-LLM Serving5 Nov

Other references · 8

  1. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 591
  2. 2024ZipCache: Accurate and Efficient KV Cache Quantization with Salient Token Identificationcited by 93
  3. 2025Latent Collaboration in Multi-Agent Systemscited by 73
  4. 2025Cache-to-Cache: Direct Semantic Communication Between Large Language Modelscited by 61
  5. 2025Cache Me If You Must: Adaptive Key-Value Quantization for Large Language Modelscited by 24
  6. 2025Oaken: Fast and Efficient LLM Serving with Online-Offline Hybrid KV Cache Quantizationcited by 14
  7. 2025MoQAE: Mixed-Precision Quantization for Long-Context LLM Inference via Mixture of Quantization-Aware Expertscited by 12
  8. 2026Don't Waste Bits! Adaptive KV-Cache Quantization for Lightweight On-Device LLMscited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026When Does Latent Communication Pay? A Causal Audit of Relayed KV Caches in Multi-Agent LLMscited by 3

Similar, not cited

Close in wording, no citation either way

  1. 2026Agent Memory Below the Prompt: Persistent Q4 KV Cache for Multi-Agent LLM Inference on Edge Devices
  2. 2026ImpactHO: Importance-Aware KV Cache Transfer for Multi-User Edge LLM Handover
  3. 2026Prefill-Free Cross-Family KV Cache Transfer for Heterogeneous Multi-Agent LLMs
  4. 2026TokenDance: Scaling Multi-Agent LLM Serving via Collective KV Cache Sharing
  5. 2026PolyKV: A Shared Asymmetrically-Compressed KV Cache Pool for Multi-Agent LLM Inference
  6. 2025A Systematic Evaluation of On-Device LLMs: Quantization, Performance, and Resources
  7. 2026TriAxialKV: Toward Extreme Low-Precision KV-Cache Quantization for Agentic Inference Tasks
  8. 2026SARA: SLO-Aware Resource Allocation for Disaggregated Agentic LLM Services