mlsanityData to 14 Oct 2026

LatentRemedy: Efficient Remedy For Non-Prefix KV Cache Reuse in Multi-Agent Systems

arXiv 2610.18014 ↗First posted 13 Oct 2026cs.CL cs.AIcited by 0

On the map: Efficient LLM inference and serving → Efficient reasoning and agent inference

Abstract (arXiv)

Non-prefix KV cache reuse is a promising approach for improving inference efficiency in multi-agent systems as agents share the same context blocks. However, when agents have conflicting tasks, directly reusing non-prefix KV caches across agents can severely degrade model performance, causing up to a 90% drop. Existing remedies, including selective recomputation and offset-based calibration, recover a limited fraction of this loss. We argue that the problem lies not only in contaminated KV representations but also in the agent's difficulty interpreting reused context under its own instructions. To this end, we introduce LatentRemedy, which uses a few task-conditioned latent tokens to help the target agent leverage reused context without recomputing the shared block. Specifically, we use an MLP to map the last hidden state of the target prefix into a small number of latent tokens, which are placed after the reused KV block to interact with the shared context. To guide subsequent predictions toward the target task, we train the MLP through full-prefill distillation. Experiments show that LatentRemedy consistently outperforms existing reuse baselines on conflicting tasks while maintaining TTFT close to direct reuse as cache size grows. Code and data are available at https://github.com/chenzhwsysu57/kvreuse.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Efficient reasoning and agent inference, one of 12 directions in Efficient LLM inference and serving. Builds on 12 of its 19 references that are in the map. First posted on arXiv 13 Oct 2026.

Builds on

12 of 19 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025KVCOMM: Online Cross-context KV-cache Communication for Efficient LLM-based Multi-agent Systems14 Oct

Other references · 11

  1. 2025Qwen3 Technical Reportcited by 8,820
  2. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  3. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  4. 2021P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Taskscited by 689
  5. 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructionscited by 493
  6. 2024CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusioncited by 223
  7. 2017Deal or No Deal? End-to-End Learning for Negotiation Dialoguescited by 209
  8. 2020From Arguments to Key Points: Towards Automatic Argument Summarizationcited by 61
  9. 2025IHEval: Evaluating Language Models on Following the Instruction Hierarchycited by 53
  10. 2024EPIC: Efficient Position-Independent Caching for Serving Large Language Modelscited by 47
  11. 2015Automated Analysis and Prediction of Job Interview Performancecited by 15

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026KVCMAS: Efficient KV cache Correction for Shared Context in Multi-Agent Systems
  2. 2026When KV Cache Reuse Fails in Multi-Agent Systems: Cross-Candidate Interaction is Crucial for LLM Judges
  3. 2026RelayCaching: Accelerating LLM Collaboration via Decoding KV Cache Reuse
  4. 2026AgentKVShift: Efficient KV Cache Reuse for Agentic Memory Systems
  5. 2025ContextPilot: Fast Long-Context Inference via Context Reuse
  6. 2026KV-Fold: One-Step KV-Cache Recurrence for Long-Context Inference
  7. 2026ICaRus: Identical Cache Reuse for Efficient Multi Model Inference
  8. 2026ReCache: Efficient KV Cache Reuse and Compression for Tool-Augmented LLM Agents