# Improving Clinical Diagnosis with Counterfactual Multi-Agent Reasoning

arXiv: https://arxiv.org/abs/2603.27820 · HTML: https://mlsanity.com/paper/2603.27820/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning.
- Builds on 24 of its 36 references that are in the map.
- First posted on arXiv 29 Mar 2026.

## Placed in

- LLM reasoning / [Clinical and biomedical applications](https://mlsanity.com/lr/clinical-and-biomedical-applications/)
- LLM agents and tool use / [Medical and scientific domain agents](https://mlsanity.com/agents/medical-and-scientific-domain-agents/)

## Builds on (most cited first, 24 of 24)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [HealthBench: Evaluating Large Language Models Towards Improved Human Health](https://mlsanity.com/paper/2505.08775/) (2025)
- [MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoning](https://mlsanity.com/paper/2311.10537/) (2023)
- [MDAgents: An Adaptive Collaboration of LLMs for Medical Decision-Making](https://mlsanity.com/paper/2404.15155/) (2024)
- [MedReason: Eliciting Factual Medical Reasoning Steps in LLMs via Knowledge Graphs](https://mlsanity.com/paper/2504.00993/) (2025)
- [Large Language Models for Disease Diagnosis: A Scoping Review](https://mlsanity.com/paper/2409.00097/) (2024)
- [Automated Lay Language Summarization of Biomedical Scientific Reviews](https://mlsanity.com/paper/2012.12573/) (2020)
- [m1: Unleash the Potential of Test-Time Scaling for Medical Reasoning with Large Language Models](https://mlsanity.com/paper/2504.00869/) (2025)
- [Reasoning-Enhanced Healthcare Predictions with Knowledge Graph Community Retrieval](https://mlsanity.com/paper/2410.04585/) (2024)
- [Med-RLVR: Emerging Medical Reasoning from a 3B base model via reinforcement Learning](https://mlsanity.com/paper/2502.19655/) (2025)
- [MedCaseReasoning: Evaluating and learning diagnostic reasoning from clinical case reports](https://mlsanity.com/paper/2505.11733/) (2025)
- [Reverse Thinking Makes LLMs Stronger Reasoners](https://mlsanity.com/paper/2411.19865/) (2024)
- [MedAgentBoard: Benchmarking Multi-Agent Collaboration with Conventional Methods for Diverse Medical Tasks](https://mlsanity.com/paper/2505.12371/) (2025)
- [Propagation and Pitfalls: Reasoning-based Assessment of Knowledge Editing through Counterfactual Tasks](https://mlsanity.com/paper/2401.17585/) (2024)
- [ER-Reason: A Benchmark Dataset for LLM Clinical Reasoning in the Emergency Room](https://mlsanity.com/paper/2505.22919/) (2025)
- [Tree-of-Reasoning: Towards Complex Medical Diagnosis via Multi-Agent Reasoning with Evidence Tree](https://mlsanity.com/paper/2508.03038/) (2025)
- [SciPrompt: Knowledge-augmented Prompting for Fine-grained Categorization of Scientific Topics](https://mlsanity.com/paper/2410.01946/) (2024)
- [PlainQAFact: Retrieval-augmented Factual Consistency Evaluation Metric for Biomedical Plain Language Summarization](https://mlsanity.com/paper/2503.08890/) (2025)

## Built on by (most cited first, 5 of 5)

- [From Question Answering to Task Completion: A Survey on Agent System and Harness Design](https://mlsanity.com/paper/2606.20683/) (2026)
- [MedGuideX: Internalizing Decision Logic from Executable Guidelines into Large Language Models for Clinical Reasoning](https://mlsanity.com/paper/2605.26567/) (2026)
- [GuideSkill: Evolving Executable LLM Agent Skills for Guideline-Grounded Clinical Reasoning](https://mlsanity.com/paper/2607.26160/) (2026)
- [CDEG: Learning Decision-Critical Evidence for Long-Horizon Diagnostic Agents](https://mlsanity.com/paper/2608.22899/) (2026)
- [Counterfactual Self-Evolving Agents for Evidence-Grounded Reasoning](https://mlsanity.com/paper/2609.32870/) (2026)

## Abstract (arXiv)

Clinical diagnosis is a complex reasoning process in which clinicians gather evidence, form hypotheses, and test them against alternative explanations. In medical training, this reasoning is explicitly developed through counterfactual questioning--e.g., asking how a diagnosis would change if a key symptom were absent or altered--to strengthen differential diagnosis skills. As large language model (LLM)-based systems are increasingly used for diagnostic support, ensuring the interpretability of their recommendations becomes critical. However, most existing LLM-based diagnostic agents reason over fixed clinical evidence without explicitly testing how individual findings support or weaken competing diagnoses. In this work, we propose a counterfactual multi-agent diagnostic framework inspired by clinician training that makes hypothesis testing explicit and evidence-grounded. Our framework introduces counterfactual case editing to modify clinical findings and evaluate how these changes affect competing diagnoses. We further define the Counterfactual Probability Gap, a method that quantifies how strongly individual findings support a diagnosis by measuring confidence shifts under these edits. These counterfactual signals guide multi-round specialist discussions, enabling agents to challenge unsupported hypotheses, refine differential diagnoses, and produce more interpretable reasoning trajectories. Across three diagnostic benchmarks and seven LLMs, our method consistently improves diagnostic accuracy over prompting and prior multi-agent baselines, with the largest gains observed in complex and ambiguous cases. Human evaluation further indicates that our framework produces more clinically useful, reliable, and coherent reasoning. These results suggest that incorporating counterfactual evidence verification is an important step toward building reliable AI systems for clinical decision support.