mlsanityData to 14 Oct 2026

Cache-to-Cache: Direct Semantic Communication Between Large Language Models

arXiv 2510.03215 ↗First posted 3 Oct 2025cs.CL cs.LGcited by 61

On the map: Efficient LLM inference and serving → KV cache optimization

Abstract (arXiv)

Multi-LLM systems harness the complementary strengths of diverse Large Language Models, achieving performance and efficiency gains that are not attainable by a single model. In existing designs, LLMs communicate through text, forcing internal representations to be transformed into output token sequences. This process both loses rich semantic information and incurs token-by-token generation latency. Motivated by these limitations, we ask: Can LLMs communicate beyond text? Oracle experiments show that enriching the KV-Cache semantics can improve response quality without increasing cache size, supporting KV-Cache as an effective medium for inter-model communication. Thus, we propose Cache-to-Cache (C2C), a new paradigm for direct semantic communication between LLMs. C2C uses a neural network to project and fuse the source model's KV-cache with that of the target model to enable direct semantic transfer. A learnable gating mechanism selects the target layers that benefit from cache communication. Compared with text communication, C2C utilizes the deep, specialized semantics from both models, while avoiding explicit intermediate text generation. Experiments show that C2C achieves 6.4-14.2% higher average accuracy than individual models. It further outperforms the text communication paradigm by approximately 3.1-5.4%, while delivering an average 2.5x speedup in latency. Our code is available at https://github.com/thu-nics/C2C.

Key facts. Cited by 61 papers in the map (data through 14 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 41 of its 57 references that are in the map. First posted on arXiv 3 Oct 2025.

Builds on

41 of 57 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025KVLink: Accelerating Large Language Models via Efficient KV Cache Reuse21 Feb
  2. 2024KVSharer: Efficient Inference via Layer-Wise Dissimilar KV Cache Sharing24 Oct
  3. 2024A Systematic Study of Cross-Layer KV Sharing for Efficient LLM Inference18 Oct
  4. 2024Reducing Transformer Key-Value Cache Size with Cross-Layer Attention21 May
  5. 2024Layer-Condensed KV Cache for Efficient Inference of Large Language Models17 May
  6. 2024ChunkAttention: Efficient Self-Attention with Prefix-Aware KV Cache and Two-Phase Partition23 Feb

Other references · 35

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2025Qwen3 Technical Reportcited by 8,820
  3. 2025Qwen2.5-VL Technical Reportcited by 5,587
  4. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  5. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,164
  6. 2025Gemma 3 Technical Reportcited by 1,824
  7. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,808
  8. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
  9. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,395
  10. 2024Qwen2.5-Coder Technical Reportcited by 1,203
  11. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,090
  12. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 1,011
23 more
  1. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challengescited by 769
  2. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 708
  3. 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 682
  4. 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertaintycited by 578
  5. 2024Mixture-of-Agents Enhances Large Language Model Capabilitiescited by 438
  6. 2025Multi-Agent Collaboration Mechanisms: A Survey of LLMscited by 431
  7. 2023A Dynamic LLM-Powered Agent Network for Task-Oriented Agent Collaborationcited by 290
  8. 2025Reasoning with Latent Thoughts: On the Power of Looped Transformerscited by 240
  9. 2024Chain of Agents: Large Language Models Collaborating on Long-Context Taskscited by 206
  10. 2024Mooncake: A KVCache-centric Disaggregated Architecture for LLM Servingcited by 196
  11. 2024Are We Done with MMLU?cited by 194
  12. 2024GraphRouter: A Graph-based Router for LLM Selectionscited by 165
  13. 2024You Only Cache Once: Decoder-Decoder Architectures for Language Modelscited by 144
  14. 2025OWL: Optimized Workforce Learning for General Multi-Agent Assistance in Real-World Task Automationcited by 131
  15. 2023Skeleton-of-Thought: Prompting LLMs for Efficient Parallel Generationcited by 105
  16. 2025Cartridges: Lightweight and general-purpose long context representations via self-studycited by 86
  17. 2024Learning to Decode Collaboratively with Multiple Language Modelscited by 62
  18. 2025PonderLM: Pretraining Language Models to Ponder in Continuous Spacecited by 44
  19. 2024DroidSpeak: KV Cache Sharing for Cross-LLM Communication and Multi-LLM Servingcited by 35
  20. 2025CITER: Collaborative Inference for Efficient Large Language Model Decoding with Token-Level Routingcited by 30
  21. 2025R2R: Efficiently Navigating Divergent Reasoning Paths with Small-Large Model Token Routingcited by 29
  22. 2024Fast and Slow Generating: An Empirical Study on Large and Small Language Models Collaborative Decodingcited by 13
  23. 2025Cognify: Supercharging Gen-AI Workflows With Hierarchical Autotuningcited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 58.

61 papers.

Sort:

By area:

  1. 2025Latent Collaboration in Multi-Agent Systemscited by 73
  2. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 50
  3. 2026The Latent Space: Foundation, Evolution, Mechanism, Ability, and Outlookcited by 43
  4. 2025Enabling Agents to Communicate Entirely in Latent Spacecited by 34
  5. 2026LatentMem: Customizing Latent Memory for Multi-Agent Systemscited by 15
  6. 2026Cross-Model KV Cache Transfer in LLM Families: A Closed-Form Linear Mapping for Prefill Reusecited by 15
  7. 2026Agent Primitives: Reusable Latent Building Blocks for Multi-Agent Systemscited by 14
  8. 2026Dual Latent Memory for Visual Multi-agent Systemcited by 10
  9. 2026LRAgent: Efficient KV Cache Sharing for Multi-LoRA LLM Agentscited by 10
  10. 2026Recursive Multi-Agent Systemscited by 10
  11. 2026The Vision Wormhole: Latent-Space Communication in Heterogeneous Multi-Agent Systemscited by 9
  12. 2026Latent Cache Flow: Model-to-Model Communication Without Textcited by 8
38 more
  1. 2026Beyond tokens: a unified framework for latent communication in LLM-based multi-agent systemscited by 7
  2. 2025Slm-mux: Orchestrating small language models for reasoningcited by 6
  3. 2026LCGuard: Latent Communication Guard for Safe KV Sharing in Multi-Agent Systemscited by 6
  4. 2026CacheBridge: Efficient Cross-Model KV Cache Transfercited by 6
  5. 2026Semantic Cache Distillation: Efficient State Transfer via Reuse and Selective Patchingcited by 5
  6. 2026StateBridge: Training-free Hidden-state Alignment for Latent Communication in LLM Multi-Agent Systemscited by 5
  7. 2026OxyGen: Unified KV Cache Management for VLA Inference under Multi-Task Parallelismcited by 4
  8. 2026Scaling Video Understanding via Compact Latent Multi-Agent Collaborationcited by 4
  9. 2026HyLaT: Efficient Multi-Agent Communication via Hybrid Latent-Text Protocolcited by 4
  10. 2026When Latent Agents Lie: KV-Cache Integrity in Multi-Agent LLM Collaborationcited by 4
  11. 2026KV Cache Translation across Heterogeneous Large Language Modelscited by 4
  12. 2026KV-Skill: Forging Expertise in the Model's Native Languagecited by 4
  13. 2026TopoDIM: One-shot Topology Generation of Diverse Interaction Modes for Multi-Agent Systemscited by 3
  14. 2026When Does Latent Communication Pay? A Causal Audit of Relayed KV Caches in Multi-Agent LLMscited by 3
  15. 2026XBridge: Entity-Grounded Latent Bridge for Heterogeneous LLM Communicationcited by 3
  16. 2026Learning to Communicate: Toward End-to-End Optimization of Multi-Agent Language Systemscited by 2
  17. 2026The Bicameral Model: Bidirectional Hidden-State Coupling Between Parallel Language Modelscited by 2
  18. 2026LLM Benchmark Datasets Should Be Contamination-Resistantcited by 2
  19. 2026LatentDx: Latent Multi-Agent Communication for Cross-Hospital Rare-Disease Diagnosiscited by 2
  20. 2026Efficient Remote KV Cache Reuse with GPU-native Video Codeccited by 1
  21. 2026QKVShare: Quantized KV-Cache Handoff for Multi-Agent On-Device LLMscited by 1
  22. 2026KVShareArena: KV-Cache Reuse Across Contexts and Model Checkpointscited by 1
  23. 2026When Less is Enough: Efficient Inference via Collaborative Reasoningcited by 0
  24. 2026LACO: Adaptive Latent Communication for Collaborative Drivingcited by 0
  25. 2026Streaming Communication in Multi-Agent Reasoningcited by 0
  26. 2026Closing the Loop on Latent Reasoning via Test-Time Reconstructioncited by 0
  27. 2026Towards Direct Latent-Space Synthesis for Parallel Branches in LLM-Agent Workflowscited by 0
  28. 2026Tandem Reinforcement Learning with Verifiable Rewardscited by 0
  29. 2026MaSRead: Content-Addressed Reading of Replicated Latent Storescited by 0
  30. 2026MentorPulse: Refreshing Cross-Model Latent Guidance for Long-Form Generationcited by 0
  31. 2026Unified AI Gateway: A Framework for Joint Model Routing and KV Cache Managementcited by 0
  32. 2026From Semantic to Token Communication: The Next Paradigm for Large-Model-Driven 6G Intelligent Connectivitycited by 0
  33. 2026OrchSLM: Probing the Dynamics of Small Language Model Orchestrationcited by 0
  34. 2026Machine-Interpretable Information: Compiling Documents into Searchable and Readable Protocol Statescited by 0
  35. 2026LatentPort: Beyond KV Cache - Cross-Model Transfer of Recurrent Memory in Hybrid Language Models: A 4B-to-9B Hybrid-State Handoff Without Target Prefix Replaycited by 0
  36. 2026Receiver-Conditioned Latent Communication gives 94% CacheBackcited by 0
  37. 2026Prefill-Free Cross-Family KV Cache Transfer for Heterogeneous Multi-Agent LLMscited by 0
  38. 2026Positions Are Not Facts: The Mismatch Between KV Caches and Memorycited by 0

50 most cited of 61 All 61 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025IC-Cache: Efficient Large Language Model Serving via In-context Caching
  2. 2026Latent Space Communication via K-V Cache Alignment
  3. 2026KV Cache Optimization Strategies for Scalable and Efficient LLM Inference
  4. 2024Keep the Cost Down: A Review on Methods to Optimize LLM' s KV-Cache Consumption
  5. 2024Beyond KV Caching: Shared Attention for Efficient LLMs
  6. 2026Towards Efficient Large Language Model Serving: A Survey on System-Aware KV Cache Optimization
  7. 2024MiniCache: KV Cache Compression in Depth Dimension for Large Language Models
  8. 2024A Survey on Large Language Model Acceleration based on KV Cache Management