mlsanityData to 7 Oct 2026

DiReCT: Diagnostic Reasoning for Clinical Notes via Large Language Models

arXiv 2408.01933 ↗First posted 4 Aug 2024cs.CL cs.AIcited by 14

On the map: LLM reasoning → Clinical and biomedical applications

Abstract (arXiv)

Large language models (LLMs) have recently showcased remarkable capabilities, spanning a wide range of tasks and applications, including those in the medical domain. Models like GPT-4 excel in medical question answering but may face challenges in the lack of interpretability when handling complex tasks in real clinical settings. We thus introduce the diagnostic reasoning dataset for clinical notes (DiReCT), aiming at evaluating the reasoning ability and interpretability of LLMs compared to human doctors. It contains 511 clinical notes, each meticulously annotated by physicians, detailing the diagnostic reasoning process from observations in a clinical note to the final diagnosis. Additionally, a diagnostic knowledge graph is provided to offer essential knowledge for reasoning, which may not be covered in the training data of existing LLMs. Evaluations of leading LLMs on DiReCT bring out a significant gap between their reasoning ability and that of human doctors, highlighting the critical need for models that can reason effectively in real-world clinical scenarios.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning. Builds on 13 of its 81 references that are in the map. First posted on arXiv 4 Aug 2024.

Builds on

13 of 81 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoning16 Nov

Other references · 12

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023Mistral 7Bcited by 3,065
  3. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  4. 2023A Survey of Large Language Modelscited by 2,537
  5. 2023PaLM 2 Technical Reportcited by 1,154
  6. 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 630
  7. 2023Zephyr: Direct Distillation of LM Alignmentcited by 498
  8. 2023MedAlpaca -- An Open-Source Collection of Medical Conversational AI Models and Training Datacited by 282
  9. 2024Benchmarking Large Language Models on Answering and Explaining Challenging Medical Questionscited by 106
  10. 2023Leveraging Medical Knowledge Graphs Into Large Language Models for Diagnosis Prediction: Design and Application Studycited by 29
  11. 2023ExplainCPE: A Free-text Explanation Benchmark of Chinese Pharmacist Examinationcited by 10
  12. 2023SemEval-2023 Task 7: Multi-Evidence Natural Language Inference for Clinical Trial Datacited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 9; 2026: 5.

14 papers.

Sort:

By area:

  1. 2025ER-Reason: A Benchmark Dataset for LLM Clinical Reasoning in the Emergency Roomcited by 21
  2. 2025Uncertainty-Aware Large Language Models for Explainable Disease Diagnosiscited by 9
  3. 2025Why Chain of Thought Fails in Clinical Text Understandingcited by 8
  4. 2025FailureSensorIQ: A Multi-Choice QA Dataset for Understanding Sensor Relationships and Failure Modescited by 6
  5. 2026Benchmarking Multi-turn Medical Diagnosis: Hold, Lure, and Self-Correctioncited by 4
  6. 2025E-InMeMo: Enhanced Prompting for Visual In-Context Learningcited by 1
  7. 2025MedCoAct: Confidence-Aware Multi-Agent Collaboration for Complete Clinical Decisioncited by 1
  8. 2025Multi-Modal Feature Fusion for Spatial Morphology Analysis of Traditional Villages via Hierarchical Graph Neural Networkscited by 1
  9. 2025CMQCIC-Bench: A Chinese Benchmark for Evaluating Large Language Models in Medical Quality Control Indicator Calculationcited by 0
  10. 2025Treatment, evidence, imitation, and chatcited by 0
  11. 2026Disentangling Deception and Hallucination Failures in LLMscited by 0
  12. 2026BioDivergence: A Benchmark and Evaluation Framework for Hidden Contextual Contradictions in Biomedical Abstractscited by 0
2 more
  1. 2026Neurosymbolic Clinical Trial Matching via LLM-Driven Abduction and Logical Verificationcited by 0
  2. 2026SMARtCARE: Privacy-Preserving Agentic AI Systems for Bounded-Autonomy Clinical Decision Supportcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Diagnostic Reasoning Prompts Reveal the Potential for Large Language Model Interpretability in Medicine
  2. 2025MedCaseReasoning: Evaluating and learning diagnostic reasoning from clinical case reports
  3. 2026Medical Reasoning with Large Language Models: A Survey and MR-Bench
  4. 2026A rubric landscape for evaluating clinical reasoning in large language models: what exists, what is missing, and what needs to be combined
  5. 2025Evaluating Hierarchical Clinical Document Classification Using Reasoning-Based LLMs
  6. 2026Clinical Reasoning Graphs: Structured Evaluation of LLM Diagnostic Reasoning Reveals Competence Without Consistency
  7. 2026Aligning Clinical Needs and AI Capabilities: A Survey on LLMs for Medical Reasoning
  8. 2025ClinicalGPT-R1: Pushing reasoning capability of generalist disease diagnosis with large language model