# Large Language Models are Clinical Reasoners: Reasoning-Aware Diagnosis Framework with Prompt-Generated Rationales

arXiv: https://arxiv.org/abs/2312.07399 · HTML: https://mlsanity.com/paper/2312.07399/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning.
- Builds on 17 of its 44 references that are in the map.
- First posted on arXiv 12 Dec 2023.

## Placed in

- LLM reasoning / [Clinical and biomedical applications](https://mlsanity.com/lr/clinical-and-biomedical-applications/)

## Builds on (most cited first, 17 of 17)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [Multimodal Chain-of-Thought Reasoning in Language Models](https://mlsanity.com/paper/2302.00923/) (2023)
- [Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes](https://mlsanity.com/paper/2305.02301/) (2023)
- [Multimodal Few-Shot Learning with Frozen Language Models](https://mlsanity.com/paper/2106.13884/) (2021)
- [Learning Spatio-Temporal Features with 3D Residual Networks for Action Recognition](https://mlsanity.com/paper/1708.07632/) (2017)
- [What do we need to build explainable AI systems for the medical domain?](https://mlsanity.com/paper/1712.09923/) (2017)
- [Clinical Reading Comprehension: A Thorough Analysis of the emrQA Dataset](https://mlsanity.com/paper/2005.00574/) (2020)
- [How Additional Knowledge can Improve Natural Language Commonsense Question Answering?](https://mlsanity.com/paper/1909.08855/) (2019)
- [Chain of Thought Prompting Elicits Knowledge Augmentation](https://mlsanity.com/paper/2307.01640/) (2023)
- [CHARD: Clinical Health-Aware Reasoning Across Dimensions for Text Generation Models](https://mlsanity.com/paper/2210.04191/) (2022)

## Built on by (most cited first, 30 of 33)

- [Two Tales of Persona in LLMs: A Survey of Role-Playing and Personalization](https://mlsanity.com/paper/2406.01171/) (2024)
- [Towards Lifelong Dialogue Agents via Timeline-based Memory Management](https://mlsanity.com/paper/2406.10996/) (2024)
- [Limitations of Large Language Models in Clinical Problem-Solving Arising from Inflexible Reasoning](https://mlsanity.com/paper/2502.04381/) (2025)
- [Prompt engineering paradigms for medical applications: scoping review and recommendations for better practices](https://mlsanity.com/paper/2405.01249/) (2024)
- [Cross-Refine: Improving Natural Language Explanation Generation by Learning in Tandem](https://mlsanity.com/paper/2409.07123/) (2024)
- [Uncertainty-Aware Large Language Models for Explainable Disease Diagnosis](https://mlsanity.com/paper/2505.03467/) (2025)
- [CliBench: A Multifaceted and Multigranular Evaluation of Large Language Models for Clinical Decision Making](https://mlsanity.com/paper/2406.09923/) (2024)
- [MedReflect: Teaching Medical LLMs to Self-Improve via Reflective Correction](https://mlsanity.com/paper/2510.03687/) (2025)
- [MACD: Multi-Agent Clinical Diagnosis with Self-Learned Knowledge for LLM](https://mlsanity.com/paper/2509.20067/) (2025)
- [PersonaFuse: A Personality Activation-Driven Framework for Enhancing Human-LLM Interactions](https://mlsanity.com/paper/2509.07370/) (2025)
- [Interpretable Differential Diagnosis with Dual-Inference Large Language Models](https://mlsanity.com/paper/2407.07330/) (2024)
- [Dialogue is Better Than Monologue: Instructing Medical LLMs via Strategical Conversations](https://mlsanity.com/paper/2501.17860/) (2025)
- [Large Language Models for Bioinformatics](https://mlsanity.com/paper/2501.06271/) (2025)
- [LLMs in Disease Diagnosis: A Comparative Study of DeepSeek-R1 and O3 Mini Across Chronic Health Conditions](https://mlsanity.com/paper/2503.10486/) (2025)
- [Can Reasoning LLMs Enhance Clinical Document Classification?](https://mlsanity.com/paper/2504.08040/) (2025)
- [Critique of Impure Reason: Unveiling the reasoning behaviour of medical Large Language Models](https://mlsanity.com/paper/2412.15748/) (2024)
- [Integrating Large Language Models with Human Expertise for Disease Detection in Electronic Health Records](https://mlsanity.com/paper/2504.00053/) (2025)
- [Learning to reason about rare diseases through retrieval-augmented agents](https://mlsanity.com/paper/2511.04720/) (2025)
- [Prompting Large Language Models for Supporting the Differential Diagnosis of Anemia](https://mlsanity.com/paper/2409.15377/) (2024)
- [MedCoAct: Confidence-Aware Multi-Agent Collaboration for Complete Clinical Decision](https://mlsanity.com/paper/2510.10461/) (2025)
- [ReSS: Learning Reasoning Models for Tabular Data Prediction via Symbolic Scaffold](https://mlsanity.com/paper/2604.13392/) (2026)
- [Improving Medical VQA through Trajectory-Aware Process Supervision](https://mlsanity.com/paper/2605.04064/) (2026)
- [D-NLP at SemEval-2024 Task 2: Evaluating Clinical Inference Capabilities of Large Language Models](https://mlsanity.com/paper/2405.04170/) (2024)
- [Can human clinical rationales improve the performance and explainability of clinical text classification models?](https://mlsanity.com/paper/2507.21302/) (2025)
- [Enhancing Trustworthy Clinical Diagnosis Decision-Making in Large Language Models via Etiology-Aware Attention Supervision](https://mlsanity.com/paper/2508.00285/) (2025)
- [SwasthLLM: a Unified Cross-Lingual, Multi-Task, and Meta-Learning Zero-Shot Framework for Medical Diagnosis Using Contrastive Representations](https://mlsanity.com/paper/2509.20567/) (2025)
- [Leveraging Evidence-Guided LLMs to Enhance Trustworthy Depression Diagnosis](https://mlsanity.com/paper/2511.17947/) (2025)
- [Advances in LLM Reasoning Enable Flexibility in Clinical Problem-Solving](https://mlsanity.com/paper/2601.11866/) (2026)
- [CONDESION-BENCH: Conditional Decision-Making of Large Language Models in Compositional Action Space](https://mlsanity.com/paper/2604.09029/) (2026)
- [Surrogate modeling for interpreting black-box LLMs in medical predictions](https://mlsanity.com/paper/2604.20331/) (2026)

## Abstract (arXiv)

Machine reasoning has made great progress in recent years owing to large language models (LLMs). In the clinical domain, however, most NLP-driven projects mainly focus on clinical classification or reading comprehension, and under-explore clinical reasoning for disease diagnosis due to the expensive rationale annotation with clinicians. In this work, we present a "reasoning-aware" diagnosis framework that rationalizes the diagnostic process via prompt-based learning in a time- and labor-efficient manner, and learns to reason over the prompt-generated rationales. Specifically, we address the clinical reasoning for disease diagnosis, where the LLM generates diagnostic rationales providing its insight on presented patient data and the reasoning path towards the diagnosis, namely Clinical Chain-of-Thought (Clinical CoT). We empirically demonstrate LLMs/LMs' ability of clinical reasoning via extensive experiments and analyses on both rationale generation and disease diagnosis in various settings. We further propose a novel set of criteria for evaluating machine-generated rationales' potential for real-world clinical settings, facilitating and benefiting future research in this area.