mlsanityData to 14 Oct 2026

Right Diagnoses, Decorative Reasoning:A Perturbation Audit of Medical Chain-of-Thought

arXiv 2608.24790 ↗First posted 25 Aug 2026cs.AIcited by 1

On the map: LLM reasoning → Clinical and biomedical applications

Abstract (arXiv)

Clinicians read chain-of-thought (CoT) rationales as evidence of medical reasoning, but whether the visible chain plays that role is rarely tested. General-domain CoT-faithfulness probes ignore clinical cost, and medical LLM evaluations treat the chain as a black box. We close this gap with a medical perturbation audit: a 30-operator battery edits both the chain and the question with clinically motivated operators (severity reversal, negation flip, demographic swap, evidence ablation), paired with a chain-update times answer-flip joint analysis that classifies each model by its failure mode. Applied to 14 LLMs on four medical QA benchmarks, three independent tests converge: the Chain-Decoupling Rate (CDR; chain does not register the edit and the answer does not flip) is 72.9% panel-wide on clinically meaningful destructive edits, chain corruption leaves accuracy unchanged, and removing CoT prompting does not reduce accuracy. Two board-certified clinicians re-annotate N=197 perturbed questions; 98.5% leave the gold defensible. The pattern holds across medical and reasoning fine-tuning and scale; on the closed-source tier, where the chain text is unavailable, the answer-side signals are consistent with the same decoupling. Our framework and CDR provide a reusable yardstick for auditing whether medical CoT is faithful or merely documentation.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning. Builds on 7 of its 37 references that are in the map. First posted on arXiv 25 Aug 2026.

Builds on

7 of 37 references.

References · 7

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2024Qwen2.5 Technical Reportcited by 4,942
  4. 2023Mistral 7Bcited by 3,164
  5. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  6. 2023Measuring Faithfulness in Chain-of-Thought Reasoningcited by 703
  7. 2023MEDITRON-70B: Scaling Medical Pretraining for Large Language Modelscited by 274

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026Better Accuracies, Worse Reasoning: A Step-Level Audit of Medical Chain-of-Thought Distillation
  2. 2025Why Chain of Thought Fails in Clinical Text Understanding
  3. 2026Faithful or Just Plausible? Evaluating the Faithfulness of Closed-Source LLMs in Medical Reasoning
  4. 2026To Reason or Not to: Selective Chain-of-Thought in Medical Question Answering
  5. 2026Step-CoT: Stepwise Visual Chain-of-Thought for Medical Visual Question Answering
  6. 2025Evaluating Reasoning Faithfulness in Medical Vision-Language Models using Multimodal Perturbations
  7. 2026CLExEval: A Human-in-the-Loop Framework for Qualitative Evaluation of LLM Clinical Reasoning
  8. 2026Better Eyes, Better Thoughts: Why Vision Chain-of-Thought Fails in Medicine