# Why Chain of Thought Fails in Clinical Text Understanding

arXiv: https://arxiv.org/abs/2509.21933 · HTML: https://mlsanity.com/paper/2509.21933/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning.
- Builds on 22 of its 42 references that are in the map.
- First posted on arXiv 26 Sep 2025.

## Placed in

- LLM reasoning / [Clinical and biomedical applications](https://mlsanity.com/lr/clinical-and-biomedical-applications/)

## Builds on (most cited first, 22 of 22)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023)
- [A Survey on LLM-as-a-Judge](https://mlsanity.com/paper/2411.15594/) (2024)
- [Least-to-Most Prompting Enables Complex Reasoning in Large Language Models](https://mlsanity.com/paper/2205.10625/) (2022)
- [Graph of Thoughts: Solving Elaborate Problems with Large Language Models](https://mlsanity.com/paper/2308.09687/) (2023)
- [Faithful Chain-of-Thought Reasoning](https://mlsanity.com/paper/2301.13379/) (2023)
- [Can large language models reason about medical questions?](https://mlsanity.com/paper/2207.08143/) (2022)
- [MedCalc-Bench: Evaluating Large Language Models for Medical Calculations](https://mlsanity.com/paper/2406.12036/) (2024)
- [Clinical Outcome Prediction from Admission Notes using Self-Supervised Knowledge Integration](https://mlsanity.com/paper/2102.04110/) (2021)
- [Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens](https://mlsanity.com/paper/2508.01191/) (2025)
- [Exploring the Numerical Reasoning Capabilities of Language Models: A Comprehensive Analysis on Tabular Data](https://mlsanity.com/paper/2311.02216/) (2023)
- [Few shot chain-of-thought driven reasoning to prompt LLMs for open ended medical question answering](https://mlsanity.com/paper/2403.04890/) (2024)
- [Scattertext: a Browser-Based Tool for Visualizing how Corpora Differ](https://mlsanity.com/paper/1703.00565/) (2017)
- [Guiding Clinical Reasoning with Large Language Models via Knowledge Seeds](https://mlsanity.com/paper/2403.06609/) (2024)
- [DiReCT: Diagnostic Reasoning for Clinical Notes via Large Language Models](https://mlsanity.com/paper/2408.01933/) (2024)
- [BRIDGE: Benchmarking Large Language Models for Understanding Real-world Clinical Practice Text](https://mlsanity.com/paper/2504.19467/) (2025)
- [Large Language Models Leverage External Knowledge to Extend Clinical Insight Beyond Language Boundaries](https://mlsanity.com/paper/2305.10163/) (2023)

## Built on by (most cited first, 9 of 9)

- [MedPRMBench: A Fine-grained Benchmark for Process Reward Models in Medical Reasoning](https://mlsanity.com/paper/2604.17282/) (2026)
- [When Reasoning Hurts: Source-Aware Evaluation of Frontier LLMs for Clinical SOAP Note Generation](https://mlsanity.com/paper/2605.24902/) (2026)
- [Better Accuracies, Worse Reasoning: A Step-Level Audit of Medical Chain-of-Thought Distillation](https://mlsanity.com/paper/2605.28301/) (2026)
- [Evaluating Metrics for Safety with LLM-as-Judges](https://mlsanity.com/paper/2512.15617/) (2025)
- [EvalQReason: A Framework for Step-Level Reasoning Evaluation in Large Language Models](https://mlsanity.com/paper/2602.02295/) (2026)
- [One Panel Does Not Fit All: Case-Adaptive Multi-Agent Deliberation for Clinical Prediction](https://mlsanity.com/paper/2604.00085/) (2026)
- [Let LLMs Judge Each Other: Multi-Agent Peer-Reviewed Reasoning for Medical Question Answering](https://mlsanity.com/paper/2606.15419/) (2026)
- [ASCRIBE: Atomic and Significance-Based Reasoning for Thai Clinical SOAP Note Generation](https://mlsanity.com/paper/2610.01234/) (2026)
- [Hindsight-Guided Rationale Distillation for Rare Disease Diagnosis](https://mlsanity.com/paper/2610.03176/) (2026)

## Abstract (arXiv)

Large language models (LLMs) are increasingly being applied to clinical care, a domain where both accuracy and transparent reasoning are critical for safe and trustworthy deployment. Chain-of-thought (CoT) prompting, which elicits step-by-step reasoning, has demonstrated improvements in performance and interpretability across a wide range of tasks. However, its effectiveness in clinical contexts remains largely unexplored, particularly in the context of electronic health records (EHRs), the primary source of clinical documentation, which are often lengthy, fragmented, and noisy. In this work, we present the first large-scale systematic study of CoT for clinical text understanding. We assess 95 advanced LLMs on 87 real-world clinical text tasks, covering 9 languages and 8 task types. Contrary to prior findings in other domains, we observe that 86.3% of models suffer consistent performance degradation in the CoT setting. More capable models remain relatively robust, while weaker ones suffer substantial declines. To better characterize these effects, we perform fine-grained analyses of reasoning length, medical concept alignment, and error profiles, leveraging both LLM-as-a-judge evaluation and clinical expert evaluation. Our results uncover systematic patterns in when and why CoT fails in clinical contexts, which highlight a critical paradox: CoT enhances interpretability but may undermine reliability in clinical text tasks. This work provides an empirical basis for clinical reasoning strategies of LLMs, highlighting the need for transparent and trustworthy approaches.