mlsanityData to 14 Oct 2026

Knowledge or Reasoning? A Close Look at How LLMs Think Across Domains

arXiv 2506.02126 ↗First posted 2 Jun 2025cs.CLcited by 14

On the map: LLM reasoning → Clinical and biomedical applications

Abstract (arXiv)

Recent advances in reasoning-enhanced Large Language Models such as OpenAI-o1/3 and DeepSeek-R1 have significantly improved performance on complex tasks. However, the quality and transparency of their internal reasoning processes remain underexplored. This work moves beyond the final-answer accuracy and investigates step-by-step reasoning in the medical and mathematical domains by explicitly decomposing the thinking trajectories into two parts: knowledge and reasoning. Specifically, we introduce a fine-grained evaluation framework that judges: (1) the correctness of knowledge used (measured by Knowledge Index (KI)) and (2) the quality of reasoning (measured by Information Gain (InfoGain)). Using this framework, we study R1-distilled and base Qwen models trained with supervised fine-tuning (SFT) and/or reinforcement learning (RL) in the medical and math domains. Three intriguing findings emerge: (1) The general reasoning abilities in R1-distilled models do not transfer effectively to the medical domain through either SFT or RL. (2) SFT raises final-answer accuracy in both domains, but often at the cost of reasoning quality: InfoGain drops by 38.9% on average compared with untrained models; In the medical domain, however, SFT remains crucial because domain knowledge is indispensable. (3) RL enhances medical reasoning by pruning inaccurate or irrelevant knowledge from reasoning paths, thereby improving both reasoning accuracy and knowledge correctness.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning. Builds on 30 of its 49 references that are in the map. First posted on arXiv 2 Jun 2025.

Builds on

30 of 49 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025MedReason: Eliciting Factual Medical Reasoning Steps in LLMs via Knowledge Graphs1 Apr
  2. 2025m1: Unleash the Potential of Test-Time Scaling for Medical Reasoning with Large Language Models1 Apr
  3. 2024HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMs25 Dec

Other references · 27

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  3. 2024GPT-4o System Cardcited by 4,588
  4. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  5. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,392
  6. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,356
  7. 2025s1: Simple test-time scalingcited by 1,341
  8. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,210
  9. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,090
  10. 2025SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-trainingcited by 672
  11. 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 638
  12. 2025LIMO: Less is More for Reasoningcited by 498
15 more
  1. 2025L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learningcited by 384
  2. 2024Large Language Models for Mathematical Reasoning: Progresses and Challengescited by 284
  3. 2025MedXpertQA: Benchmarking Expert-Level Medical Reasoning and Understandingcited by 224
  4. 2025SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Modelscited by 212
  5. 2022ROSCOE: A Suite of Metrics for Scoring Step-by-Step Reasoningcited by 177
  6. 2025Seed1.5-Thinking: Advancing Superb Reasoning Models with Reinforcement Learningcited by 161
  7. 2025Llama-Nemotron: Efficient Reasoning Modelscited by 158
  8. 2023Verify-and-Edit: A Knowledge-Enhanced Chain-of-Thought Frameworkcited by 139
  9. 2023ReCEval: Evaluating Reasoning Chains via Correctness and Informativenesscited by 77
  10. 2023Temporal Data Meets LLM -- Explainable Financial Time Series Forecastingcited by 77
  11. 2025STAR-1: Safer Alignment of Reasoning LLMs with 1K Datacited by 68
  12. 2023RCOT: Detecting and Rectifying Factual Inconsistency in Reasoning by Reversing Chain-of-Thoughtcited by 47
  13. 2024A Preliminary Study of o1 in Medicine: Are We Closer to an AI Doctor?cited by 33
  14. 2024Chain of Logic: Rule-Based Reasoning with Large Language Modelscited by 18
  15. 2024Benchmarking Large Language Models for Math Reasoning Taskscited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 8.

14 papers.

Sort:

By area:

  1. 2025What Characterizes Effective Reasoning? Revisiting Length, Review, and Structure of CoTcited by 25
  2. 2025Reasoning or Retrieval? A Study of Answer Attribution on Large Reasoning Modelscited by 10
  3. 2025Tracing the Traces: Latent Temporal Signals for Efficient and Accurate Reasoningcited by 9
  4. 2025Correct Reasoning Paths Visit Shared Decision Pivotscited by 5
  5. 2026Learning from Mistakes: Negative Reasoning Samples Enhance Out-of-Domain Generalizationcited by 5
  6. 2025Demystifying Scientific Problem-Solving in LLMs by Probing Knowledge and Reasoningcited by 4
  7. 2026ClinSeekAgent: Automating Multimodal Evidence Seeking for Agentic Clinical Reasoningcited by 4
  8. 2025Towards Efficient Medical Reasoning with Minimal Fine-Tuning Datacited by 3
  9. 2026Reasoning Fails Where Step Flow Breakscited by 2
  10. 2026Prescriptive Scaling Reveals the Evolution of Language Model Capabilitiescited by 1
  11. 2026TARo: Token-level Adaptive Routing for LLM Test-time Alignmentcited by 1
  12. 2026Spatiotemporal Hidden-State Dynamics as a Signature of Internal Reasoning in Large Language Modelscited by 0
2 more
  1. 2026CausalMix: Data Mixture as Causal Inference for Language Model Trainingcited by 0
  2. 2026WordPolo: Evaluating Language Models Through Iterative Semantic Feedbackcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Reasoning LLMs in the Medical Domain: A Literature Survey
  2. 2025Disentangling Reasoning and Knowledge in Medical Large Language Models
  3. 2025Medical Reasoning in the Era of LLMs: A Systematic Review of Enhancement Techniques and Applications
  4. 2026Advances in LLM Reasoning Enable Flexibility in Clinical Problem-Solving
  5. 2025From General Reasoning to Domain Expertise: Uncovering the Limits of Generalization in Large Language Models
  6. 2025Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain Perspective
  7. 2025Automating Expert-Level Medical Reasoning Evaluation of Large Language Models
  8. 2025Decoupling Knowledge and Reasoning in LLMs: An Exploration Using Cognitive Dual-System Theory