mlsanityData to 7 Oct 2026

Diagnostic Reasoning Prompts Reveal the Potential for Large Language Model Interpretability in Medicine

arXiv 2308.06834 ↗First posted 13 Aug 2023cs.CL cs.AI cs.HCcited by 83

On the map: LLM reasoning → Clinical and biomedical applications

Abstract (arXiv)

One of the major barriers to using large language models (LLMs) in medicine is the perception they use uninterpretable methods to make clinical decisions that are inherently different from the cognitive processes of clinicians. In this manuscript we develop novel diagnostic reasoning prompts to study whether LLMs can perform clinical reasoning to accurately form a diagnosis. We find that GPT4 can be prompted to mimic the common clinical reasoning processes of clinicians without sacrificing diagnostic accuracy. This is significant because an LLM that can use clinical reasoning to provide an interpretable rationale offers physicians a means to evaluate whether LLMs can be trusted for patient care. Novel prompting methods have the potential to expose the black box of LLMs, bringing them one step closer to safe and effective use in medicine.

Key facts. Cited by 83 papers in the map (data through 7 Oct 2026). Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning. Builds on 11 of its 34 references that are in the map. First posted on arXiv 13 Aug 2023.

Builds on

11 of 34 references.

References · 11

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  4. 2023Let's Verify Step by Stepcited by 3,598
  5. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  6. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,324
  7. 2023Instruction Tuning with GPT-4cited by 676
  8. 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 630
  9. 2023Towards Expert-Level Medical Question Answering with Large Language Modelscited by 382
  10. 2022Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLPcited by 316
  11. 2023Prompt Engineering for Healthcare: Methodologies and Applicationscited by 52

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 24; 2025: 41; 2026: 15.

83 papers.

Sort:

By area:

  1. 2023Large language models in healthcare and medical domain: A reviewcited by 126
  2. 2024Medical Graph RAG: Towards Safe Medical Large Language Model via Graph Retrieval-Augmented Generationcited by 89
  3. 2024MEDEC: A Benchmark for Medical Error Detection and Correction in Clinical Notescited by 77
  4. 2025Evaluating Step-by-step Reasoning Traces: A Surveycited by 62
  5. 2024Improving Retrieval-Augmented Generation in Medicine with Iterative Follow-up Questionscited by 54
  6. 2024CoD, Towards an Interpretable Medical Agent using Chain of Diagnosiscited by 43
  7. 2025Medical Large Language Model Benchmarks Should Prioritize Construct Validitycited by 36
  8. 2025MedCaseReasoning: Evaluating and learning diagnostic reasoning from clinical case reportscited by 34
  9. 2024Superhuman performance of a large language model on the reasoning tasks of a physiciancited by 28
  10. 2025MEDDxAgent: A Unified Modular Agent Framework for Explainable Automatic Differential Diagnosiscited by 23
  11. 2025Hyper-RAG: Combating LLM Hallucinations using Hypergraph-Driven Retrieval-Augmented Generationcited by 23
  12. 2025Automating Expert-Level Medical Reasoning Evaluation of Large Language Modelscited by 21
38 more
  1. 2024ArgMed-Agents: Explainable Clinical Decision Reasoning with LLM Disscusion via Argumentation Schemescited by 13
  2. 2024Prompt engineering paradigms for medical applications: scoping review and recommendations for better practicescited by 13
  3. 2025BRIDGE: Benchmarking Large Language Models for Understanding Real-world Clinical Practice Textcited by 11
  4. 2024Me LLaMA: Foundation Large Language Models for Medical Applicationscited by 10
  5. 2025A Survey of Scaling in Large Language Model Reasoningcited by 10
  6. 2024Large Language Models in Biomedical and Health Informatics: A Review with Bibliometric Analysiscited by 9
  7. 2025Uncertainty-Aware Large Language Models for Explainable Disease Diagnosiscited by 9
  8. 2025Evaluating Reasoning Faithfulness in Medical Vision-Language Models using Multimodal Perturbationscited by 9
  9. 2024Generalizable and Scalable Multistage Biomedical Concept Normalization Leveraging Large Language Modelscited by 7
  10. 2024When Reasoning Meets Information Aggregation: A Case Study with Sports Narrativescited by 7
  11. 2025Integrating Chain-of-Thought and Retrieval Augmented Generation Enhances Rare Disease Diagnosis from Clinical Notescited by 7
  12. 2025ClinicalGPT-R1: Pushing reasoning capability of generalist disease diagnosis with large language modelcited by 7
  13. 2025MedKGI: Iterative Differential Diagnosis with Medical Knowledge Graphs and Information-Guided Inquiringcited by 7
  14. 2025MACD: Multi-Agent Clinical Diagnosis with Self-Learned Knowledge for LLMcited by 6
  15. 2024Beyond Accuracy: Investigating Error Types in GPT-4 Responses to USMLE Questionscited by 5
  16. 2024Fine Tuning Large Language Models for Medicine: The Role and Importance of Direct Preference Optimizationcited by 5
  17. 2024Best Practices for Large Language Models in Radiologycited by 5
  18. 2024Large Language Models for Medical Forecasting -- Foresight 2cited by 5
  19. 2025MedPAIR: Measuring Physicians and AI Relevance Alignment in Medical Question Answeringcited by 5
  20. 2023Methods to Estimate Large Language Model Confidencecited by 4
  21. 2024Interpretable Differential Diagnosis with Dual-Inference Large Language Modelscited by 4
  22. 2025MedM-VL: What Makes a Good Medical LVLM?cited by 4
  23. 2025Timely Clinical Diagnosis through Active Test Selectioncited by 4
  24. 2026Retrieval-Augmented Agentic Rubric Generation for Reliable Medical Response Evaluationcited by 4
  25. 2026Quantifying and Mitigating Premature Closure in Frontier LLMscited by 3
  26. 2023The Tyranny of Possibilities in the Design of Task-Oriented LLM Systems: A Scoping Surveycited by 2
  27. 2024XAI4LLM. Let Machine Learning Models and LLMs Collaborate for Enhanced In-Context Learning in Healthcarecited by 2
  28. 2024Pareto-Optimized Open-Source LLMs for Healthcare via Context Retrievalcited by 2
  29. 2024Position Paper On Diagnostic Uncertainty Estimation from Large Language Models: Next-Word Probability Is Not Pre-test Probabilitycited by 2
  30. 2024Critique of Impure Reason: Unveiling the reasoning behaviour of medical Large Language Modelscited by 2
  31. 2025Zero-Shot Cognitive Impairment Detection from Speech Using AudioLLMcited by 2
  32. 2025Evaluating Hierarchical Clinical Document Classification Using Reasoning-Based LLMscited by 2
  33. 2025Learning to reason about rare diseases through retrieval-augmented agentscited by 2
  34. 2026Small Agent Group is the Future of Digital Healthcited by 2
  35. 2026Emulating Clinician Cognition via Self-Evolving Deep Clinical Researchcited by 2
  36. 2024Prompting Large Language Models for Supporting the Differential Diagnosis of Anemiacited by 1
  37. 2025Explainable Multimodal Depression Recognition in Clinical Interviews via PHQ-Aligned Symptom Summarizationcited by 1
  38. 2025OphthBench: A Comprehensive Benchmark for Evaluating Large Language Models in Chinese Ophthalmologycited by 1

50 most cited of 83 All 83 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Prompting language influences diagnostic reasoning and accuracy of large language models
  2. 2023Large Language Models are Clinical Reasoners: Reasoning-Aware Diagnosis Framework with Prompt-Generated Rationales
  3. 2026Medical Reasoning with Large Language Models: A Survey and MR-Bench
  4. 2023Large Language Models Perform Diagnostic Reasoning
  5. 2024DiReCT: Diagnostic Reasoning for Clinical Notes via Large Language Models
  6. 2025Reasoning LLMs in the Medical Domain: A Literature Survey
  7. 2026"Do I Trust the AI?" Towards Trustworthy AI-Assisted Diagnosis: Understanding User Perception in LLM-Supported Reasoning
  8. 2024Guiding Clinical Reasoning with Large Language Models via Knowledge Seeds