# Enabling Agents to Communicate Entirely in Latent Space

arXiv: https://arxiv.org/abs/2511.09149 · HTML: https://mlsanity.com/paper/2511.09149/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 31 of its 46 references that are in the map.
- First posted on arXiv 12 Nov 2025.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 30 of 31)

- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning](https://mlsanity.com/paper/2307.08691/) (2023)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [ALFWorld: Aligning Text and Embodied Environments for Interactive Learning](https://mlsanity.com/paper/2010.03768/) (2020)
- [Training Large Language Models to Reason in a Continuous Latent Space](https://mlsanity.com/paper/2412.06769/) (2024)
- [Why Do Multi-Agent LLM Systems Fail?](https://mlsanity.com/paper/2503.13657/) (2025)
- [AFlow: Automating Agentic Workflow Generation](https://mlsanity.com/paper/2410.10762/) (2024)
- [Multi-Agent Collaboration Mechanisms: A Survey of LLMs](https://mlsanity.com/paper/2501.06322/) (2025)
- [Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach](https://mlsanity.com/paper/2502.05171/) (2025)
- [Reasoning Models Don't Always Say What They Think](https://mlsanity.com/paper/2505.05410/) (2025)
- [Think before you speak: Training Language Models With Pause Tokens](https://mlsanity.com/paper/2310.02226/) (2023)
- [CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillation](https://mlsanity.com/paper/2502.21074/) (2025)
- [Compressed Chain of Thought: Efficient Reasoning Through Dense Representations](https://mlsanity.com/paper/2412.13171/) (2024)
- [Trial and Error: Exploration-Based Trajectory Optimization for LLM Agents](https://mlsanity.com/paper/2403.02502/) (2024)
- [Let's Think Dot by Dot: Hidden Computation in Transformer Language Models](https://mlsanity.com/paper/2404.15758/) (2024)
- [Multi-Agent Cooperation and the Emergence of (Natural) Language](https://mlsanity.com/paper/1612.07182/) (2016)
- [MultiAgentBench: Evaluating the Collaboration and Competition of LLM agents](https://mlsanity.com/paper/2503.01935/) (2025)
- [Cut the Crap: An Economical Communication Pipeline for LLM-based Multi-Agent Systems](https://mlsanity.com/paper/2410.02506/) (2024)
- [Emergence of Linguistic Communication from Referential Games with Symbolic and Pixel Input](https://mlsanity.com/paper/1804.03984/) (2018)
- [A Survey on Latent Reasoning](https://mlsanity.com/paper/2507.06203/) (2025)
- [Relaxed Recursive Transformers: Effective Parameter Sharing with Layer-wise LoRA](https://mlsanity.com/paper/2410.20672/) (2024)
- [Cache-to-Cache: Direct Semantic Communication Between Large Language Models](https://mlsanity.com/paper/2510.03215/) (2025)
- [Let Models Speak Ciphers: Multiagent Debate through Embeddings](https://mlsanity.com/paper/2310.06272/) (2023)
- [EvoAgentX: An Automated Framework for Evolving Agentic Workflows](https://mlsanity.com/paper/2507.03616/) (2025)
- [Thought Communication in Multiagent Collaboration](https://mlsanity.com/paper/2510.20733/) (2025)
- [Deliberation in Latent Space via Differentiable Cache Augmentation](https://mlsanity.com/paper/2412.17747/) (2024)
- [Communicating Activations Between Language Model Agents](https://mlsanity.com/paper/2501.14082/) (2025)
- [LLM-Based Multi-Agent Blackboard System for Information Discovery in Data Science](https://mlsanity.com/paper/2510.01285/) (2025)
- [Augmenting Multi-Agent Communication with State Delta Trajectory](https://mlsanity.com/paper/2506.19209/) (2025)

## Built on by (most cited first, 30 of 34)

- [A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agents](https://mlsanity.com/paper/2601.12538/) (2026)
- [The Latent Space: Foundation, Evolution, Mechanism, Ability, and Outlook](https://mlsanity.com/paper/2604.02029/) (2026)
- [Recursive Multi-Agent Systems](https://mlsanity.com/paper/2604.25917/) (2026)
- [The Vision Wormhole: Latent-Space Communication in Heterogeneous Multi-Agent Systems](https://mlsanity.com/paper/2602.15382/) (2026)
- [Beyond tokens: a unified framework for latent communication in LLM-based multi-agent systems](https://mlsanity.com/paper/2606.05711/) (2026)
- [LCGuard: Latent Communication Guard for Safe KV Sharing in Multi-Agent Systems](https://mlsanity.com/paper/2605.22786/) (2026)
- [StateBridge: Training-free Hidden-state Alignment for Latent Communication in LLM Multi-Agent Systems](https://mlsanity.com/paper/2608.13317/) (2026)
- [HyLaT: Efficient Multi-Agent Communication via Hybrid Latent-Text Protocol](https://mlsanity.com/paper/2605.25421/) (2026)
- [Out of Sight, Not Out of Mind: Unveiling Latent Attack in Latent-based Multi-Agent Systems](https://mlsanity.com/paper/2605.28214/) (2026)
- [When Latent Agents Lie: KV-Cache Integrity in Multi-Agent LLM Collaboration](https://mlsanity.com/paper/2606.28958/) (2026)
- [KV Cache Translation across Heterogeneous Large Language Models](https://mlsanity.com/paper/2607.28979/) (2026)
- [Representational Collapse in Multi-Agent LLM Committees: Measurement and Diversity-Aware Consensus](https://mlsanity.com/paper/2604.03809/) (2026)
- [When Does Latent Communication Pay? A Causal Audit of Relayed KV Caches in Multi-Agent LLMs](https://mlsanity.com/paper/2608.04893/) (2026)
- [Dual-Cache Latent Space Communication between Heterogeneous Language Models](https://mlsanity.com/paper/2608.20617/) (2026)
- [Learning to Communicate: Toward End-to-End Optimization of Multi-Agent Language Systems](https://mlsanity.com/paper/2604.21794/) (2026)
- [The Bicameral Model: Bidirectional Hidden-State Coupling Between Parallel Language Models](https://mlsanity.com/paper/2605.11167/) (2026)
- [A Token/KV-Cache Communication Media Selection and Resource Allocation Strategy for Multi-Agent Collaboration](https://mlsanity.com/paper/2605.25422/) (2026)
- [LatentDx: Latent Multi-Agent Communication for Cross-Hospital Rare-Disease Diagnosis](https://mlsanity.com/paper/2606.13945/) (2026)
- [Large Language Models Do Not Always Need Readable Language](https://mlsanity.com/paper/2606.19857/) (2026)
- [Confidence Laundering in Agent Systems: Why Uncertainty Needs a Latent Carrier](https://mlsanity.com/paper/2606.20662/) (2026)
- [Position: Introspective Experience from Conversational Environments as a Path to Better Learning](https://mlsanity.com/paper/2602.14910/) (2026)
- [Good Agentic Friends Do Not Just Give Verbal Advice: They Can Update Your Weights](https://mlsanity.com/paper/2605.13839/) (2026)
- [LACO: Adaptive Latent Communication for Collaborative Driving](https://mlsanity.com/paper/2605.22504/) (2026)
- [Towards Direct Latent-Space Synthesis for Parallel Branches in LLM-Agent Workflows](https://mlsanity.com/paper/2606.14672/) (2026)
- [Engineering Trustworthy Agentic AI for Critical Systems](https://mlsanity.com/paper/2607.18548/) (2026)
- [How Far Do Simple Transformations Translate Across Text Embedding Models?](https://mlsanity.com/paper/2608.05980/) (2026)
- [MaSRead: Content-Addressed Reading of Replicated Latent Stores](https://mlsanity.com/paper/2608.11218/) (2026)
- [AeroLat: Channel-Aware Latent Space Semantic Communication for Decentralized UAV Swarms](https://mlsanity.com/paper/2609.16947/) (2026)
- [Receiver-Conditioned Latent Communication gives 94% CacheBack](https://mlsanity.com/paper/2609.32046/) (2026)
- [Draft-KV: Learning Useful Latent Communication Between Language Models](https://mlsanity.com/paper/2609.34754/) (2026)

## Abstract (arXiv)

While natural language is the de facto communication medium for LLM-based agents, it presents a fundamental constraint. The process of downsampling rich, internal latent states into discrete tokens inherently limits the depth and nuance of information that can be transmitted, thereby hindering collaborative problem-solving. Inspired by telepathy, which bypasses symbolic language in communication, we propose Interlat (Inter-agent Latent Space Communication), a paradigm that leverages the continuous last hidden states of an LLM as a representation of its thought for direct communication (termed latent communication). An additional learned compression process further compresses latent communication via latent space reasoning. Experiments demonstrate that Interlat outperforms both fine-tuned chain-of-thought (CoT) prompting and single-agent baselines, even across heterogeneous models, promoting more exploratory behavior and enabling genuine utilization of latent information. Further compression not only substantially accelerates inference by up to 24 times but also maintains competitive performance through an efficient information-preserving mechanism. We position this work as a feasibility study of entirely latent space inter-agent communication, and our results highlight its potential, offering valuable insights for future research. Our code is available at https://github.com/XiaoDu-flying/Interlat.