mlsanityData to 7 Oct 2026

When KV Cache Reuse Fails in Multi-Agent Systems: Cross-Candidate Interaction is Crucial for LLM Judges

arXiv 2601.08343 ↗First posted 13 Jan 2026cs.MA cs.CLcited by 2

On the map: LLM agents and tool use → Software engineering coding agents

also in 1 more
Abstract (arXiv)

Multi-agent LLM systems routinely generate multiple candidate responses that are aggregated by an LLM judge. To reduce the dominant prefill cost in such pipelines, recent work advocates KV cache reuse across partially shared contexts and reports substantial speedups for generation agents. In this work, we show that these efficiency gains do not transfer uniformly to judge-centric inference. Across GSM8K, MMLU, and HumanEval, we find that reuse strategies that are effective for execution agents can severely perturb judge behavior: end-task accuracy may appear stable, yet the judge's selection becomes highly inconsistent with dense prefill. We quantify this risk using Judge Consistency Rate (JCR) and provide diagnostics showing that reuse systematically weakens cross-candidate attention, especially for later candidate blocks. Our ablation further demonstrates that explicit cross-candidate interaction is crucial for preserving dense-prefill decisions. Overall, our results identify a previously overlooked failure mode of KV cache reuse and highlight judge-centric inference as a distinct regime that demands dedicated, risk-aware system design.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 28 of its 46 references that are in the map. First posted on arXiv 13 Jan 2026.

Builds on

28 of 46 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025KVFlow: Efficient Prefix Caching for Accelerating LLM-Based Multi-Agent Workflows10 Jul

Other references · 27

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  5. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  6. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  7. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  8. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,615
  9. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
  10. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 873
  11. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 820
  12. 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 480
15 more
  1. 2023Large Language Models Are Not Robust Multiple Choice Selectorscited by 454
  2. 2024LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methodscited by 451
  3. 2023PIXIU: A Large Language Model, Instruction Data and Evaluation Benchmark for Financecited by 217
  4. 2024CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusioncited by 211
  5. 2024BoNBoN Alignment for Large Language Models and the Sweetness of Best-of-n Samplingcited by 120
  6. 2024Fast Best-of-N Decoding via Speculative Rejectioncited by 112
  7. 2023Split and Merge: Aligning Position Biases in LLM-based Evaluatorscited by 71
  8. 2024TurboRAG: Accelerating Retrieval-Augmented Generation with Precomputed KV Caches for Chunked Textcited by 60
  9. 2025KVLink: Accelerating Large Language Models via Efficient KV Cache Reusecited by 60
  10. 2024Constructing Domain-Specific Evaluation Sets for LLM-as-a-judgecited by 28
  11. 2024JuStRank: Benchmarking LLM Judges for System Rankingcited by 27
  12. 2024RARE: Retrieval-Augmented Reasoning Enhancement for Large Language Modelscited by 24
  13. 2023DocLens: Multi-aspect Fine-grained Evaluation for Medical Text Generationcited by 16
  14. 2025Speculative Prefill: Turbocharging TTFT with Lightweight and Training-Free Token Importance Estimationcited by 12
  15. 2025Revisiting Self-Consistency from Dynamic Distributional Alignment Perspective on Answer Aggregationcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026KV Cache Translation across Heterogeneous Large Language Modelscited by 4
  2. 2026KVShareArena: KV-Cache Reuse Across Contexts and Model Checkpointscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026RelayCaching: Accelerating LLM Collaboration via Decoding KV Cache Reuse
  2. 2026KVCMAS: Efficient KV cache Correction for Shared Context in Multi-Agent Systems
  3. 2026Persistent Memory in Multi-Agent LLM Inference: What It Costs, What It Buys, and When You Can Tell
  4. 2025KVCOMM: Online Cross-context KV-cache Communication for Efficient LLM-based Multi-agent Systems
  5. 2026TokenDance: Scaling Multi-Agent LLM Serving via Collective KV Cache Sharing
  6. 2026Learning Agent Execution for KV-Cache Management in Agentic Serving
  7. 2026ICaRus: Identical Cache Reuse for Efficient Multi Model Inference
  8. 2026Agent Memory Below the Prompt: Persistent Q4 KV Cache for Multi-Agent LLM Inference on Edge Devices