# Beyond Direct Diagnosis: LLM-based Multi-Specialist Agent Consultation for Automatic Diagnosis

arXiv: https://arxiv.org/abs/2401.16107 · HTML: https://mlsanity.com/paper/2401.16107/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Placed in Medical and scientific domain agents, one of 16 directions in LLM agents and tool use.
- Builds on 17 of its 30 references that are in the map.
- First posted on arXiv 29 Jan 2024.

## Placed in

- LLM agents and tool use / [Medical and scientific domain agents](https://mlsanity.com/agents/medical-and-scientific-domain-agents/)

## Builds on (most cited first, 17 of 17)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [Baichuan 2: Open Large-scale Language Models](https://mlsanity.com/paper/2309.10305/) (2023)
- [Leveraging Large Language Models for Multiple Choice Question Answering](https://mlsanity.com/paper/2210.12353/) (2022)
- [Can large language models reason about medical questions?](https://mlsanity.com/paper/2207.08143/) (2022)
- [HuaTuo: Tuning LLaMA Model with Chinese Medical Knowledge](https://mlsanity.com/paper/2304.06975/) (2023)
- [GeneGPT: Augmenting Large Language Models with Domain Tools for Improved Access to Biomedical Information](https://mlsanity.com/paper/2304.09667/) (2023)
- [DISC-MedLLM: Bridging General Large Language Models and Real-World Medical Consultation](https://mlsanity.com/paper/2308.14346/) (2023)
- [End-to-End Knowledge-Routed Relational Dialogue System for Automatic Diagnosis](https://mlsanity.com/paper/1901.10623/) (2019)
- [Knowledge-Augmented Reasoning Distillation for Small Language Models in Knowledge-Intensive Tasks](https://mlsanity.com/paper/2305.18395/) (2023)
- [Diaformer: Automatic Diagnosis via Symptoms Sequence Generation](https://mlsanity.com/paper/2112.10433/) (2021)
- [Knowledge-tuning Large Language Models with Structured Medical Knowledge Bases for Reliable Response Generation in Chinese](https://mlsanity.com/paper/2309.04175/) (2023)
- [DxFormer: A Decoupled Automatic Diagnostic System Based on Decoder-Encoder Transformer with Dense Symptom Representations](https://mlsanity.com/paper/2205.03755/) (2022)
- [Imperfect also Deserves Reward: Multi-Level and Sequential Reward Modeling for Better Dialog Management](https://mlsanity.com/paper/2104.04748/) (2021)
- [CoAD: Automatic Diagnosis through Symptom and Disease Collaborative Generation](https://mlsanity.com/paper/2307.08290/) (2023)

## Built on by (most cited first, 23 of 23)

- [A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agents](https://mlsanity.com/paper/2601.12538/) (2026)
- [A Survey on Medical Large Language Models: Technology, Application, Trustworthiness, and Future Directions](https://mlsanity.com/paper/2406.03712/) (2024)
- ['Simulacrum of Stories': Examining Large Language Models as Qualitative Research Participants](https://mlsanity.com/paper/2409.19430/) (2024)
- [DrHouse: An LLM-empowered Diagnostic Reasoning System through Harnessing Outcomes from Sensor Data and Expert Knowledge](https://mlsanity.com/paper/2405.12541/) (2024)
- [Agentic AI Systems Applied to tasks in Financial Services: Modeling and model risk management crews](https://mlsanity.com/paper/2502.05439/) (2025)
- [Evolving Medical Imaging Agents via Experience-driven Self-skill Discovery](https://mlsanity.com/paper/2603.05860/) (2026)
- [MedGUIDE: Benchmarking Clinical Decision-Making in Large Language Models](https://mlsanity.com/paper/2505.11613/) (2025)
- [WSI-Agents: A Collaborative Multi-Agent System for Multi-Modal Whole Slide Image Analysis](https://mlsanity.com/paper/2507.14680/) (2025)
- [Large Language Models for Interpretable Mental Health Diagnosis](https://mlsanity.com/paper/2501.07653/) (2025)
- [MoodAngels: A Retrieval-augmented Multi-agent Framework for Psychiatry Diagnosis](https://mlsanity.com/paper/2506.03750/) (2025)
- [LungNoduleAgent: A Collaborative Multi-Agent System for Precision Diagnosis of Lung Nodules](https://mlsanity.com/paper/2511.21042/) (2025)
- [QCAgent: An agentic framework for quality-controllable pathology report generation from whole slide image](https://mlsanity.com/paper/2603.01647/) (2026)
- [Beyond Frameworks: Unpacking Collaboration Strategies in Multi-Agent Systems](https://mlsanity.com/paper/2505.12467/) (2025)
- [Auditing Multimodal LLM Raters: Central Tendency Bias in Clinical Ordinal Scoring](https://mlsanity.com/paper/2605.16386/) (2026)
- [DALL-M: Context-Aware Clinical Data Augmentation with LLMs](https://mlsanity.com/paper/2407.08227/) (2024)
- [Silence is Not Consensus: Disrupting Agreement Bias in Multi-Agent LLMs via Catfish Agent for Clinical Decision Making](https://mlsanity.com/paper/2505.21503/) (2025)
- [Multi-Agent Reasoning with Consistency Verification Improves Uncertainty Calibration in Medical MCQA](https://mlsanity.com/paper/2603.24481/) (2026)
- [Agentic AI in Healthcare & Medicine: A Seven-Dimensional Taxonomy for Empirical Evaluation of LLM-based Agents](https://mlsanity.com/paper/2602.04813/) (2026)
- [A Safety-Aware Role-Orchestrated Multi-Agent LLM Framework for Behavioral Health Communication Simulation](https://mlsanity.com/paper/2604.00249/) (2026)
- [When Pigs Get Sick: Multi-Agent AI for Swine Disease Detection](https://mlsanity.com/paper/2503.15204/) (2025)
- [LLM-Driven Medical Document Analysis: Enhancing Trustworthy Pathology and Differential Diagnosis](https://mlsanity.com/paper/2506.19702/) (2025)
- [LLM-Enabled Multi-Agent Systems: Empirical Evaluation and Insights into Emerging Design Patterns & Paradigms](https://mlsanity.com/paper/2601.03328/) (2026)
- [LingLan: An Advancing Traditional Chinese Medicine Diagnosis LLM with Multimodal Data](https://mlsanity.com/paper/2609.25715/) (2026)

## Abstract (arXiv)

Automatic diagnosis is a significant application of AI in healthcare, where diagnoses are generated based on the symptom description of patients. Previous works have approached this task directly by modeling the relationship between the normalized symptoms and all possible diseases. However, in the clinical diagnostic process, patients are initially consulted by a general practitioner and, if necessary, referred to specialists in specific domains for a more comprehensive evaluation. The final diagnosis often emerges from a collaborative consultation among medical specialist groups. Recently, large language models have shown impressive capabilities in natural language understanding. In this study, we adopt tuning-free LLM-based agents as medical practitioners and propose the Agent-derived Multi-Specialist Consultation (AMSC) framework to model the diagnosis process in the real world by adaptively fusing probability distributions of agents over potential diseases. Experimental results demonstrate the superiority of our approach compared with baselines. Notably, our approach requires significantly less parameter updating and training time, enhancing efficiency and practical utility. Furthermore, we delve into a novel perspective on the role of implicit symptoms within the context of automatic diagnosis.