mlsanityData to 14 Oct 2026

Better Eyes, Better Thoughts: Why Vision Chain-of-Thought Fails in Medicine

arXiv 2603.06665 ↗First posted 2 Mar 2026cs.CV cs.AIcited by 4

On the map: LLM reasoning → Clinical and biomedical applications

also in 1 more
Abstract (arXiv)

Large vision-language models (VLMs) often benefit from chain-of-thought (CoT) prompting in general domains, yet its efficacy in medical vision-language tasks remains underexplored. We report a counter-intuitive trend: on medical visual question answering, CoT frequently underperforms direct answering (DirA) across general-purpose and medical-specific models. We attribute this to a medical perception bottleneck: subtle, domain-specific cues can weaken visual grounding, and CoT may compound early perceptual uncertainty rather than correct it. To probe this hypothesis, we introduce two training-free, inference-time grounding interventions: (i) perception anchoring via region-of-interest cues and (ii) description grounding via high-quality textual guidance. Across multiple benchmarks and model families, these interventions improve accuracy, mitigate CoT degradation, and in several settings reverse the CoT--DirA inversion. Our findings suggest that reliable clinical VLMs require robust visual grounding and cross-modal alignment, beyond extending text-driven reasoning chains. Code is available hrefhttps://github.com/TianYin123/BetterEyesBetterThoughtshere.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning. Builds on 24 of its 28 references that are in the map. First posted on arXiv 2 Mar 2026.

Builds on

24 of 28 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Think Twice to See More: Iterative Visual Reasoning in Medical VLMs11 Oct
  2. 2025Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoning8 Jun
  3. 2024MedCoT: Medical Chain of Thought via Hierarchical Expert18 Dec

Other references · 21

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  4. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  5. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
  6. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 838
  7. 2016A Diagram Is Worth A Dozen Imagescited by 804
  8. 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 550
  9. 2026Kimi K2.5: Visual Agentic Intelligencecited by 501
  10. 2020PathVQA: 30000+ Questions for Medical Visual Question Answeringcited by 475
  11. 2021SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answeringcited by 449
  12. 2023PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answeringcited by 338
9 more
  1. 2022Learning to Prompt for Open-Vocabulary Object Detection with Vision-Language Modelcited by 317
  2. 2024OmniMedVQA: A New Large-Scale Comprehensive Evaluation Benchmark for Medical LVLMcited by 184
  3. 2023DDCoT: Duty-Distinct Chain-of-Thought Prompting for Multimodal Reasoning in Language Modelscited by 128
  4. 2025Hulu-Med: A Transparent Generalist Model towards Holistic Medical Vision-Language Understandingcited by 102
  5. 2025Unified Multimodal Chain-of-Thought Reward Model through Reinforcement Fine-Tuningcited by 90
  6. 2025HSSBench: Benchmarking Humanities and Social Sciences Ability for Multimodal Large Language Modelscited by 17
  7. 2025V2T-CoT: From Vision to Text Chain-of-Thought for Medical Reasoning and Diagnosiscited by 15
  8. 2026Unveiling and Bridging the Functional Perception Gap in MLLMs: Atomic Visual Alignment and Hierarchical Evaluation via PET-Benchcited by 3
  9. 2025Can Large Reasoning Models Improve Accuracy on Mathematical Tasks Using Flawed Thinking?cited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026MedSynapse-V: Bridging Visual Perception and Clinical Intuition via Latent Memory Evolutioncited by 12
  2. 2026Eyes on VLM: Benchmarking Gaze Following and Social Gaze Prediction in Vision Language Modelscited by 0
  3. 2026Visually Grounded Self-Reflection for Vision-Language Models via Reinforcement Learningcited by 0
  4. 2026EVADE: Evidence-Verified Agentic Diagnosis with Escapecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026To Agree or To Be Right? The Grounding-Sycophancy Tradeoff in Medical Vision-Language Models
  2. 2025Are Large Vision Language Models Truly Grounded in Medical Images? Evidence from Italian Clinical Visual Question Answering
  3. 2025S-Chain: Structured Visual Chain-of-Thought For Medicine
  4. 2025CoTBox-TTT: Grounding Medical VQA with Visual Chain-of-Thought Boxes During Test-time Training
  5. 2026Do Medical Vision Language Models Actually See? A Counterfactual Grounding Framework and Hard-Negative Contrastive Training for Visually-Reliant Medical VLMs
  6. 2026Why Does Grounding Hurt Medical VQA? Benchmarking, Diagnosis, and Fine-Tuning of Vision-Language Models
  7. 2026When Chain-of-Thought Backfires: Evaluating Prompt Sensitivity in Medical Language Models
  8. 2026ClinCoT: Clinical-Aware Visual Chain-of-Thought for Medical Vision Language Models