# When Latent Agents Lie: KV-Cache Integrity in Multi-Agent LLM Collaboration arXiv: https://arxiv.org/abs/2606.28958 ยท HTML: https://mlsanity.com/paper/2606.28958/ ## Key facts - Cited by 4 papers in the map (data through 14 Oct 2026). - Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. - Builds on 34 of its 41 references that are in the map. - First posted on arXiv 27 Jun 2026. ## Placed in - LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/) ## Builds on (most cited first, 30 of 34) - [Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates](https://mlsanity.com/paper/1803.01498/) (2018) - [DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving](https://mlsanity.com/paper/2401.09670/) (2024) - [Why Do Multi-Agent LLM Systems Fail?](https://mlsanity.com/paper/2503.13657/) (2025) - [The Hidden Vulnerability of Distributed Learning in Byzantium](https://mlsanity.com/paper/1802.07927/) (2018) - [Robust Aggregation for Federated Learning](https://mlsanity.com/paper/1912.13445/) (2019) - [A Little Is Enough: Circumventing Defenses For Distributed Learning](https://mlsanity.com/paper/1902.06156/) (2019) - [Prompt Infection: LLM-to-LLM Prompt Injection within Multi-Agent Systems](https://mlsanity.com/paper/2410.07283/) (2024) - [Latent Collaboration in Multi-Agent Systems](https://mlsanity.com/paper/2511.20639/) (2025) - [Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents](https://mlsanity.com/paper/2505.02077/) (2025) - [Cache-to-Cache: Direct Semantic Communication Between Large Language Models](https://mlsanity.com/paper/2510.03215/) (2025) - [Let Models Speak Ciphers: Multiagent Debate through Embeddings](https://mlsanity.com/paper/2310.06272/) (2023) - [Language Models are Injective and Hence Invertible](https://mlsanity.com/paper/2510.15511/) (2025) - [Enabling Agents to Communicate Entirely in Latent Space](https://mlsanity.com/paper/2511.09149/) (2025) - [TAMAS: Benchmarking Adversarial Risks in Multi-Agent LLM Systems](https://mlsanity.com/paper/2511.05269/) (2025) - [AgentLeak: A Benchmark for Internal-Channel Privacy Leakage in Multi-Agent LLM Systems](https://mlsanity.com/paper/2602.11510/) (2026) - [C-FedRAG: A Confidential Federated Retrieval-Augmented Generation System](https://mlsanity.com/paper/2412.13163/) (2024) - [Systematic Failures in Collective Reasoning under Distributed Information in Multi-Agent LLMs](https://mlsanity.com/paper/2505.11556/) (2025) - [Latent Space Communication via K-V Cache Alignment](https://mlsanity.com/paper/2601.06123/) (2026) - [Augmenting Multi-Agent Communication with State Delta Trajectory](https://mlsanity.com/paper/2506.19209/) (2025) - [Efficient Multi-Agent Collaboration with Tool Use for Online Planning in Complex Table Question Answering](https://mlsanity.com/paper/2412.20145/) (2024) - [Information Leakage from Embedding in Large Language Models](https://mlsanity.com/paper/2405.11916/) (2024) - [An Adversary-Resistant Multi-Agent LLM System via Credibility Scoring](https://mlsanity.com/paper/2505.24239/) (2025) - [MAD-Spear: A Conformity-Driven Prompt Injection Attack on Multi-Agent Debate Systems](https://mlsanity.com/paper/2507.13038/) (2025) - [The Vision Wormhole: Latent-Space Communication in Heterogeneous Multi-Agent Systems](https://mlsanity.com/paper/2602.15382/) (2026) - [SentinelNet: Safeguarding Multi-Agent Collaboration Through Credit-Based Dynamic Threat Detection](https://mlsanity.com/paper/2510.16219/) (2025) - [LCGuard: Latent Communication Guard for Safe KV Sharing in Multi-Agent Systems](https://mlsanity.com/paper/2605.22786/) (2026) - [RAGentA: Multi-Agent Retrieval-Augmented Generation for Attributed Question Answering](https://mlsanity.com/paper/2506.16988/) (2025) - [HyLaT: Efficient Multi-Agent Communication via Hybrid Latent-Text Protocol](https://mlsanity.com/paper/2605.25421/) (2026) - [Out of Sight, Not Out of Mind: Unveiling Latent Attack in Latent-based Multi-Agent Systems](https://mlsanity.com/paper/2605.28214/) (2026) - [When Embedding-Based Defenses Fail: Rethinking Safety in LLM-Based Multi-Agent Systems](https://mlsanity.com/paper/2605.01133/) (2026) ## Built on by (most cited first, 4 of 4) - [When Does Latent Communication Pay? A Causal Audit of Relayed KV Caches in Multi-Agent LLMs](https://mlsanity.com/paper/2608.04893/) (2026) - [SoK: When Safe Agents Fail Together: The Security of Multi Agent LLM Systems](https://mlsanity.com/paper/2609.00595/) (2026) - [When Goals Leak Across Relationships: How Cached Goal Representations Shape Cross-Client Decisions](https://mlsanity.com/paper/2610.17365/) (2026) - [Hybrid Reasoning Agents](https://mlsanity.com/paper/2610.18861/) (2026) ## Abstract (arXiv) LLM agents can share more than text. In some systems, an agent can send a short visible message while also passing its full KV-cache state to another model. This hidden state can help the final model combine evidence from several agents, but it is also hard to inspect. A visible message may look harmless even if the hidden state has been changed. We study this problem in a multi-agent question-answering setup. Specialists each see part of the evidence, send a short commitment, and pass full KV-cache state to a coordinator. In clean runs, this latent collaboration improves over a matched text-only version. On transformed HiddenBench with Qwen3-4B, it reaches EM/F1 of 0.338/0.486, compared with 0.231/0.369 for text collaboration. Qwen3-8B and HotPotQA runs show the same direction of improvement. The problem appears when one specialist is malicious. Some false visible commitments can steer answers. More seriously, changing the hidden KV state can collapse performance even when the visible commitment still looks plausible. A verifier that checks only text misses this failure mode. Simple magnitude checks catch some obvious corruptions, but adaptive attacks can evade them while still damaging the final answer. The most reliable fix we find is not to guess whether hidden state looks normal, but to protect it in transport. We implement an HMAC-SHA256 manifest that binds the specialist, session, model, visible commitment, tensor metadata, and payload digest. It accepts all 774 honest replayed payloads and rejects all 295 recorded tampered payloads. The main lesson is that full-KV latent memory can be useful, but it should be treated as a security-sensitive object, not as ordinary internal model state.