# LatentDx: Latent Multi-Agent Communication for Cross-Hospital Rare-Disease Diagnosis

arXiv: https://arxiv.org/abs/2606.13945 · HTML: https://mlsanity.com/paper/2606.13945/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Medical and scientific domain agents, one of 16 directions in LLM agents and tool use.
- Builds on 24 of its 40 references that are in the map.
- First posted on arXiv 11 Jun 2026.

## Placed in

- LLM agents and tool use / [Medical and scientific domain agents](https://mlsanity.com/agents/medical-and-scientific-domain-agents/)

## Builds on (most cited first, 24 of 24)

- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [Efficient Streaming Language Models with Attention Sinks](https://mlsanity.com/paper/2309.17453/) (2023)
- [Training Large Language Models to Reason in a Continuous Latent Space](https://mlsanity.com/paper/2412.06769/) (2024)
- [Learning to Compress Prompts with Gist Tokens](https://mlsanity.com/paper/2304.08467/) (2023)
- [MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoning](https://mlsanity.com/paper/2311.10537/) (2023)
- [Think before you speak: Training Language Models With Pause Tokens](https://mlsanity.com/paper/2310.02226/) (2023)
- [Agent Hospital: A Simulacrum of Hospital with Evolvable Medical Agents](https://mlsanity.com/paper/2405.02957/) (2024)
- [CacheGen: KV Cache Compression and Streaming for Fast Large Language Model Serving](https://mlsanity.com/paper/2310.07240/) (2023)
- [Text Embeddings Reveal (Almost) As Much As Text](https://mlsanity.com/paper/2310.06816/) (2023)
- [Towards Building the Federated GPT: Federated Instruction Tuning](https://mlsanity.com/paper/2305.05644/) (2023)
- [OpenFedLLM: Training Large Language Models on Decentralized Private Data via Federated Learning](https://mlsanity.com/paper/2402.06954/) (2024)
- [Soft Thinking: Unlocking the Reasoning Potential of LLMs in Continuous Concept Space](https://mlsanity.com/paper/2505.15778/) (2025)
- [Federated Learning Enables Big Data for Rare Cancer Boundary Detection](https://mlsanity.com/paper/2204.10836/) (2022)
- [MedAgent-Pro: Towards Evidence-based Multi-modal Medical Diagnosis via Reasoning Agentic Workflow](https://mlsanity.com/paper/2503.18968/) (2025)
- [DP-Forward: Fine-tuning and Inference on Language Models with Differential Privacy in Forward Pass](https://mlsanity.com/paper/2309.06746/) (2023)
- [Cache-to-Cache: Direct Semantic Communication Between Large Language Models](https://mlsanity.com/paper/2510.03215/) (2025)
- [An Agentic System for Rare Disease Diagnosis with Traceable Reasoning](https://mlsanity.com/paper/2506.20430/) (2025)
- [RareBench: Can LLMs Serve as Rare Diseases Specialists?](https://mlsanity.com/paper/2402.06341/) (2024)
- [Enabling Agents to Communicate Entirely in Latent Space](https://mlsanity.com/paper/2511.09149/) (2025)
- [Depth Gives a False Sense of Privacy: LLM Internal States Inversion](https://mlsanity.com/paper/2507.16372/) (2025)
- [Latent Space Communication via K-V Cache Alignment](https://mlsanity.com/paper/2601.06123/) (2026)
- [Fed-BioMed: Open, Transparent and Trusted Federated Learning for Real-world Healthcare Applications](https://mlsanity.com/paper/2304.12012/) (2023)
- [KaVa: Latent Reasoning via Compressed KV-Cache Distillation](https://mlsanity.com/paper/2510.02312/) (2025)
- [PFID: Privacy First Inference Delegation Framework for LLMs](https://mlsanity.com/paper/2406.12238/) (2024)

## Built on by (most cited first, 2 of 2)

- [When Does Latent Communication Pay? A Causal Audit of Relayed KV Caches in Multi-Agent LLMs](https://mlsanity.com/paper/2608.04893/) (2026)
- [MedicalHarness: A Controlled Evaluation of LLMs and Agent Harnesses on Medical Tasks](https://mlsanity.com/paper/2610.05778/) (2026)

## Abstract (arXiv)

Rare diseases affect over 300 million patients across more than 7,000 conditions, yet no single hospital encounters enough cases of any one condition for reliable diagnosis. Cross-hospital collaboration could help by allowing a diagnosing institution to use distributed, case-specific diagnostic evidence, but privacy regulations restrict the transmission of identifiable clinical text across institutional boundaries. This setting raises two challenges: existing medical agent systems often rely on textual evidence exchange, while raw latent states such as hidden states and KV caches may still reveal prompt-derived clinical content. We introduce LatentDx, a latent multi-agent communication framework in which hospital agents keep private clinical records and retrieved cases local, and send compact latent KV blocks to a host agent for rare-disease diagnosis. LatentDx supports two deployment settings: same-backbone hospital agents use latent KV distillation, while hospitals with different LLM backbones use cross-family latent alignment. On CrossRare-Bench, a self-built large-scale rare-disease benchmark with hospital-level partitions, LatentDx improves cross-hospital diagnostic performance while reducing reconstructable clinical content relative to raw-latent communication baselines.