mlsanityData to 14 Oct 2026

Critique of Impure Reason: Unveiling the reasoning behaviour of medical Large Language Models

arXiv 2412.15748 ↗First posted 20 Dec 2024cs.CL cs.AI cs.LGcited by 2

On the map: LLM reasoning → Clinical and biomedical applications

Abstract (arXiv)

Background: Despite the current ubiquity of Large Language Models (LLMs) across the medical domain, there is a surprising lack of studies which address their reasoning behaviour. We emphasise the importance of understanding reasoning behaviour as opposed to high-level prediction accuracies, since it is equivalent to explainable AI (XAI) in this context. In particular, achieving XAI in medical LLMs used in the clinical domain will have a significant impact across the healthcare sector. Results: Therefore, in this work, we adapt the existing concept of reasoning behaviour and articulate its interpretation within the specific context of medical LLMs. We survey and categorise current state-of-the-art approaches for modeling and evaluating reasoning reasoning in medical LLMs. Additionally, we propose theoretical frameworks which can empower medical professionals or machine learning engineers to gain insight into the low-level reasoning operations of these previously obscure models. We also outline key open challenges facing the development of Large Reasoning Models. Conclusion: The subsequent increased transparency and trust in medical machine learning models by clinicians as well as patients will accelerate the integration, application as well as further development of medical AI for the healthcare system as a whole.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning. Builds on 54 of its 100 references that are in the map. First posted on arXiv 20 Dec 2024.

Builds on

54 of 100 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024DrHouse: An LLM-empowered Diagnostic Reasoning System through Harnessing Outcomes from Sensor Data and Expert Knowledge21 May
  2. 2024ClinicalAgent: Clinical Trial Multi-Agent System with Large Language Model-based Reasoning23 Apr
  3. 2024ArgMed-Agents: Explainable Clinical Decision Reasoning with LLM Disscusion via Argumentation Schemes10 Mar
  4. 2023Large Language Models are Clinical Reasoners: Reasoning-Aware Diagnosis Framework with Prompt-Generated Rationales12 Dec
  5. 2023Diagnostic Reasoning Prompts Reveal the Potential for Large Language Model Interpretability in Medicine13 Aug
  6. 2022DR.BENCH: Diagnostic Reasoning Benchmark for Clinical Natural Language Processing29 Sep

Other references · 48

  1. 2017Attention Is All You Needcited by 60,361
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2017Proximal Policy Optimization Algorithmscited by 15,215
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  6. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  7. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  8. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  9. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  10. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  11. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  12. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
36 more
  1. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
  2. 2017Deep reinforcement learning from human preferencescited by 3,912
  3. 2023Let's Verify Step by Stepcited by 3,789
  4. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  5. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  6. 2024Qwen2 Technical Reportcited by 2,120
  7. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  8. 2022Large Language Models Encode Clinical Knowledgecited by 1,830
  9. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,601
  10. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,356
  11. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,288
  12. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 1,039
  13. 2022Solving math word problems with process- and outcome-based feedbackcited by 901
  14. 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 797
  15. 2022MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answeringcited by 543
  16. 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 494
  17. 2023Causal Reasoning and Large Language Models: Opening a New Frontier for Causalitycited by 353
  18. 2024Chain-of-Thought Reasoning Without Promptingcited by 210
  19. 2024Marco-o1: Towards Open Reasoning Models for Open-Ended Solutionscited by 144
  20. 2020The Language Interpretability Tool: Extensible, Interactive Visualizations and Analysis for NLP Modelscited by 125
  21. 2022DDXPlus: A New Dataset For Automatic Medical Diagnosiscited by 88
  22. 2024Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Surveycited by 83
  23. 2023A Survey of Reasoning with Foundation Modelscited by 70
  24. 2024CARES: A Comprehensive Benchmark of Trustworthiness in Medical Vision Language Modelscited by 64
  25. 2024From LLM to NMT: Advancing Low-Resource Machine Translation with Claudecited by 58
  26. 2024Automatic Interactive Evaluation for Large Language Models with State Aware Patient Simulatorcited by 39
  27. 2023Interactive Natural Language Processingcited by 31
  28. 2023IfQA: A Dataset for Open-domain Question Answering under Counterfactual Presuppositionscited by 26
  29. 2024Securing Large Language Models: Addressing Bias, Misinformation, and Prompt Attackscited by 26
  30. 2023MedDM:LLM-executable clinical guidance tree for clinical decision-makingcited by 19
  31. 2023Interpretable Medical Diagnostics with Structured Data Extraction by Large Language Modelscited by 13
  32. 2024A Systematic Analysis of Large Language Models as Soft Reasoners: The Case of Syllogistic Inferencescited by 13
  33. 2024MeNTi: Bridging Medical Calculator and LLM Agent with Nested Tool Callingcited by 10
  34. 2023Applying Large Language Models for Causal Structure Learning in Non Small Cell Lung Cancercited by 8
  35. 2022Towards Trustworthy Automatic Diagnosis Systems by Emulating Doctors' Reasoning with Deep Reinforcement Learningcited by 5
  36. 2022Doctor XAvIer: Explainable Diagnosis on Physician-Patient Dialogues and XAI Evaluationcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025A Sober Look at Progress in Language Model Reasoning: Pitfalls and Paths to Reproducibilitycited by 93
  2. 2026Reforming the Mechanism: Editing Reasoning Patterns in LLMs with Circuit Reshapingcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026Medical Reasoning with Large Language Models: A Survey and MR-Bench
  2. 2026Aligning Clinical Needs and AI Capabilities: A Survey on LLMs for Medical Reasoning
  3. 2025Medical Reasoning in the Era of LLMs: A Systematic Review of Enhancement Techniques and Applications
  4. 2025Automating Expert-Level Medical Reasoning Evaluation of Large Language Models
  5. 2025Reasoning LLMs in the Medical Domain: A Literature Survey
  6. 2025Disentangling Reasoning and Knowledge in Medical Large Language Models
  7. 2025ControlMed: Adding Reasoning Control to Medical Language Model
  8. 2026Advances in LLM Reasoning Enable Flexibility in Clinical Problem-Solving