MedCLM: Learning to Localize and Reason via a CoT-Curriculum in Medical Vision-Language Models
On the map: LLM reasoning → Clinical and biomedical applications
Abstract (arXiv)
Bridging clinical diagnostic reasoning with AI remains a central challenge in medical imaging. We introduce MedCLM, an automated pipeline that converts detection datasets into large-scale medical visual question answering (VQA) data with Chain-of-Thought (CoT) reasoning by linking lesion boxes to organ segmentation and structured rationales. These contextual signals enable medical vision-language models to generate question-answer pairs with step-by-step reasoning. To utilize this data effectively, we propose an Integrated CoT-Curriculum Strategy composed of an Easy stage with explicit lesion boxes for visual grounding, a Medium stage that encourages implicit localization, and a Hard stage for weakly supervised reasoning. Experimental results demonstrate that MedCLM attains state-of-the-art performance on several medical VQA benchmarks, providing a scalable framework for developing clinically aligned medical vision-language models.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning. Builds on 11 of its 60 references that are in the map. First posted on arXiv 6 Oct 2025.
Builds on
11 of 60 references.
References · 11
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
- 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
- 2017ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseasescited by 969
- 2023Multimodal Chain-of-Thought Reasoning in Language Modelscited by 740
- 2019MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographscited by 590
- 2023PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answeringcited by 333
- 2024HuatuoGPT-Vision, Towards Injecting Medical Visual Knowledge into Multimodal LLMs at Scalecited by 194
- 2023Towards Generalist Foundation Model for Radiology by Leveraging Web-scale 2D&3D Medical Datacited by 176
- 2023PMC-LLaMA: Towards Building Open-source Language Models for Medicinecited by 87
- 2018Hide-and-Seek: A Data Augmentation Technique for Weakly-Supervised Localization and Beyondcited by 36
- 2023Accurate Fine-Grained Segmentation of Human Anatomy in Radiographs via Volumetric Pseudo-Labelingcited by 24
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025CoTBox-TTT: Grounding Medical VQA with Visual Chain-of-Thought Boxes During Test-time Training
- 2026LiteMedCoT-VL: Parameter-Efficient Adaptation for Medical Visual Question Answering
- 2026From Reasoning to Pixels: Grounded Medical Multimodal LLMs for VQA and Segmentation
- 2024MC-CoT: A Modular Collaborative CoT Framework for Zero-shot Medical-VQA with LLM and MLLM Integration
- 2026MedLVR: Latent Visual Reasoning for Reliable Medical Visual Question Answering
- 2026Step-CoT: Stepwise Visual Chain-of-Thought for Medical Visual Question Answering
- 2025S-Chain: Structured Visual Chain-of-Thought For Medicine
- 2026Why Does Grounding Hurt Medical VQA? Benchmarking, Diagnosis, and Fine-Tuning of Vision-Language Models