mlsanityData to 14 Oct 2026

Towards Accurate Differential Diagnosis with Large Language Models

arXiv 2312.00164 ↗First posted 30 Nov 2023cs.CY cs.AIcited by 116

Not on the map. Why?

Cited by 22 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

An accurate differential diagnosis (DDx) is a cornerstone of medical care, often reached through an iterative process of interpretation that combines clinical history, physical examination, investigations and procedures. Interactive interfaces powered by Large Language Models (LLMs) present new opportunities to both assist and automate aspects of this process. In this study, we introduce an LLM optimized for diagnostic reasoning, and evaluate its ability to generate a DDx alone or as an aid to clinicians. 20 clinicians evaluated 302 challenging, real-world medical cases sourced from the New England Journal of Medicine (NEJM) case reports. Each case report was read by two clinicians, who were randomized to one of two assistive conditions: either assistance from search engines and standard medical resources, or LLM assistance in addition to these tools. All clinicians provided a baseline, unassisted DDx prior to using the respective assistive tools. Our LLM for DDx exhibited standalone performance that exceeded that of unassisted clinicians (top-10 accuracy 59.1% vs 33.6%, [p = 0.04]). Comparing the two assisted study arms, the DDx quality score was higher for clinicians assisted by our LLM (top-10 accuracy 51.7%) compared to clinicians without its assistance (36.1%) (McNemar's Test: 45.7, p < 0.01) and clinicians with search (44.4%) (4.75, p = 0.03). Further, clinicians assisted by our LLM arrived at more comprehensive differential lists than those without its assistance. Our study suggests that our LLM for DDx has potential to improve clinicians' diagnostic reasoning and accuracy in challenging cases, meriting further real-world evaluation for its ability to empower physicians and widen patients' access to specialist-level expertise.

Key facts. Cited by 116 papers in the map (data through 14 Oct 2026). Builds on 15 of its 43 references that are in the map. First posted on arXiv 30 Nov 2023.

Builds on

15 of 43 references.

References · 15

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  2. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  3. 2022Large Language Models Encode Clinical Knowledgecited by 1,830
  4. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,356
  5. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,305
  6. 2023PaLM 2 Technical Reportcited by 1,162
  7. 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 638
  8. 2023Towards Generalist Biomedical AIcited by 289
  9. 2023Do Large Language Models Know What They Don't Know?cited by 243
  10. 2023Med-HALT: Medical Domain Hallucination Test for Large Language Modelscited by 157
  11. 2019A deep learning system for differential diagnosis of skin diseasescited by 118
  12. 2023Zero-Shot Goal-Directed Dialogue via RL on Imagined Conversationscited by 38
3 more
  1. 2023Generative Relevance Feedback with Large Language Modelscited by 20
  2. 2023Multimodal Foundation Models Exploit Text to Make Medical Image Predictionscited by 20
  3. 2022Active Acquisition for Multimodal Temporal Data: A Challenging Decision-Making Taskcited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 22; 2025: 58; 2026: 36.

116 papers.

Sort:

By area:

  1. 2025HealthBench: Evaluating Large Language Models Towards Improved Human Healthcited by 429
  2. 2025Agent Laboratory: Using LLM Agents as Research Assistantscited by 425
  3. 2024Capabilities of Gemini Models in Medicinecited by 237
  4. 2024MDAgents: An Adaptive Collaboration of LLMs for Medical Decision-Makingcited by 163
  5. 2025Towards a Science of Scaling Agent Systemscited by 138
  6. 2024Towards Conversational Diagnostic AIcited by 79
  7. 2024Health-LLM: Large Language Models for Health Prediction via Wearable Sensor Datacited by 66
  8. 2025Sequential Diagnosis with Language Modelscited by 60
  9. 2024The opportunities and risks of large language models in mental healthcited by 54
  10. 2025An Agentic System for Rare Disease Diagnosis with Traceable Reasoningcited by 52
  11. 2025SeRL: Self-Play Reinforcement Learning for Large Language Models with Limited Datacited by 51
  12. 2024WSI-LLaVA: A Multimodal Large Language Model for Whole Slide Imagecited by 40
38 more
  1. 2025Medical Large Language Model Benchmarks Should Prioritize Construct Validitycited by 38
  2. 2025EndoChat: Grounded Multimodal Large Language Model for Endoscopic Surgerycited by 37
  3. 2025MedCaseReasoning: Evaluating and learning diagnostic reasoning from clinical case reportscited by 36
  4. 2024Superhuman performance of a large language model on the reasoning tasks of a physiciancited by 28
  5. 2025A Novel Evaluation Benchmark for Medical LLMs: Illuminating Safety and Effectiveness in Clinical Domainscited by 28
  6. 2024Data-Centric Foundation Models in Computational Healthcare: A Surveycited by 27
  7. 2024STEER: Assessing the Economic Rationality of Large Language Modelscited by 27
  8. 2024Towards a Personal Health Large Language Modelcited by 27
  9. 2024What Are the Odds? Language Models Are Capable of Probabilistic Reasoningcited by 26
  10. 2024MEDIC: Comprehensive Evaluation of Leading Indicators for LLM Safety and Utility in Clinical Applicationscited by 25
  11. 2025Automating Expert-Level Medical Reasoning Evaluation of Large Language Modelscited by 22
  12. 2024Transforming Wearable Data into Personal Health Insights using Large Language Model Agentscited by 19
  13. 2025Working with AI: Measuring the Applicability of Generative AI to Occupationscited by 19
  14. 2025Language Agents for Hypothesis-driven Clinical Decision Making with Reinforcement Learningcited by 15
  15. 2025InfiMed-ORBIT: Aligning LLMs on Open-Ended Complex Tasks via Rubric-Based Incremental Trainingcited by 12
  16. 2025Standardization of Psychiatric Diagnoses -- Role of Fine-tuned LLM Consortium and OpenAI-gpt-oss Reasoning LLM Enabled Decision Support Systemcited by 12
  17. 2025A Scalable Framework for Evaluating Health Language Modelscited by 11
  18. 2025Proof-of-TBI -- Fine-Tuned Vision Language Model Consortium and OpenAI-o3 Reasoning LLM-Based Medical Diagnosis Support System for Mild Traumatic Brain Injury (TBI) Predictioncited by 11
  19. 2024Me LLaMA: Foundation Large Language Models for Medical Applicationscited by 10
  20. 2025End-to-End Agentic RAG System Training for Traceable Diagnostic Reasoningcited by 10
  21. 2025Uncertainty-Aware Large Language Models for Explainable Disease Diagnosiscited by 9
  22. 2024CliBench: A Multifaceted and Multigranular Evaluation of Large Language Models for Clinical Decision Makingcited by 8
  23. 2024From Text to Multimodality: Exploring the Evolution and Impact of Large Language Models in Medical Practicecited by 8
  24. 2025Tiered Agentic Oversight: A Hierarchical Multi-Agent System for Healthcare Safetycited by 8
  25. 2026ATPO: Adaptive Tree Policy Optimization for Multi-Turn Medical Dialoguecited by 8
  26. 2025Integrating Chain-of-Thought and Retrieval Augmented Generation Enhances Rare Disease Diagnosis from Clinical Notescited by 7
  27. 2025Applications of Large Models in Medicinecited by 6
  28. 2025Exploring Efficiency Frontiers of Thinking Budget in Medical Reasoning: Scaling Laws between Computational Resources and Reasoning Qualitycited by 6
  29. 2024SPEER: Sentence-Level Planning of Long Clinical Summaries via Embedded Entity Retrievalcited by 5
  30. 2024Enabling Scalable Evaluation of Bias Patterns in Medical LLMscited by 5
  31. 2025MedBioLM: Optimizing Medical and Biological QA with Fine-Tuned Large Language Models and Retrieval-Augmented Generationcited by 5
  32. 2025MedPAIR: Measuring Physicians and AI Relevance Alignment in Medical Question Answeringcited by 5
  33. 2025Memorization in Large Language Models in Medicine: Prevalence, Characteristics, and Implicationscited by 5
  34. 2025EHR-MCP: Real-world Evaluation of Clinical Information Retrieval by Large Language Models via Model Context Protocolcited by 5
  35. 2025Retrieval-Augmented Generation in Medicine: A Scoping Review of Technical Implementations, Clinical Applications, and Ethical Considerationscited by 5
  36. 2024Interpretable Differential Diagnosis with Dual-Inference Large Language Modelscited by 4
  37. 2025WiseMind: a knowledge-guided multi-agent framework for accurate and empathetic psychiatric diagnosiscited by 4
  38. 2025Insulin Resistance Prediction From Wearables and Routine Blood Biomarkerscited by 4

50 most cited of 116 All 116 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026NSIDDx: A Design Framework for Neuro-Symbolic, Practitioner-First Differential Diagnosis in Low-Resource Settings
  2. 2025MEDDxAgent: A Unified Modular Agent Framework for Explainable Automatic Differential Diagnosis
  3. 2026MultiDx: A Multi-Source Knowledge Integration Framework towards Diagnostic Reasoning
  4. 2025Improving Interactive Diagnostic Ability of a Large Language Model Agent Through Clinical Experience Learning
  5. 2025Medical Reasoning in LLMs: An In-Depth Analysis of DeepSeek R1
  6. 2024DrHouse: An LLM-empowered Diagnostic Reasoning System through Harnessing Outcomes from Sensor Data and Expert Knowledge
  7. 2026VeriLLMed: Interactive Visual Debugging of Medical Large Language Models with Knowledge Graphs
  8. 2024DiReCT: Diagnostic Reasoning for Clinical Notes via Large Language Models