mlsanityData to 14 Oct 2026

Towards Enhancing 3D Spatial Reasoning in Medical Multimodal Large Language Models

arXiv 2607.13860 ↗First posted 15 Jul 2026cs.CVcited by 0

On the map: LLM reasoning → Clinical and biomedical applications

also in 1 more
Abstract (arXiv)

While Multimodal Large Language Models (MLLMs) have demonstrated remarkable success in 2D medical image understanding, their extension to 3D volumetric imaging remains hindered by prohibitive annotation costs and dataset opacity. Current data formats, predominantly consisting of rigid Visual Question Answering (VQA) pairs or unstructured final clinical reports, typically fail to capture explicit clinical reasoning. To address this limitation, we introduce a large-scale structured reasoning dataset constructed via a novel slice-wise data synthesis paradigm. Inspired by the genuine diagnostic workflow of radiologists, this paradigm models visual cognition by decomposing the complex 3D reading process, translating global clinical priors into fine-grained, per-slice observations that are subsequently synthesized into an interpretable Chain-of-Thought (CoT). Crucially, this synthesized reasoning framework enforces essential clinical principles: sequential spatial tracking, multi-slice spatial awareness for artifact mitigation, and differential exclusion. To validate this approach, we instruction-tune a standard 2D-pretrained MLLM baseline using the synthesized data to enhance its volumetric comprehension. Comprehensive evaluations across multiple 3D medical benchmarks demonstrate that our method yields significant performance improvements over the 2D baseline. Furthermore, the resulting model exhibits robust spatial reasoning capabilities and rivals resource-intensive native 3D architectures, effectively bridging the performance gap. Ultimately, this data-centric strategy unlocks deep volumetric understanding and highly interpretable clinical logic without requiring computationally expensive 3D-specific pre-training. The complete repository, including datasets and training workflows, is publicly available at https://github.com/2020420145009/hounsfield.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning. Builds on 28 of its 33 references that are in the map. First posted on arXiv 15 Jul 2026.

Builds on

28 of 33 references.

Closest ancestors · 7

Same direction, newest first

  1. 2026Beyond Medical Diagnostics: How Medical Multimodal Large Language Models Think in Space14 Mar
  2. 2026ClinCoT: Clinical-Aware Visual Chain-of-Thought for Medical Vision Language Models1 Mar
  3. 2026M3CoTBench: Benchmark Chain-of-Thought of MLLMs in Medical Image Understanding13 Jan
  4. 2025Fleming-VL: Towards Universal Medical Visual Reasoning with Multimodal LLMs2 Nov
  5. 2025Fleming-R1: Toward Expert-Level Medical Reasoning via Reinforcement Learning18 Sep
  6. 2025Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoning8 Jun
  7. 2024MedCoT: Medical Chain of Thought via Hierarchical Expert18 Dec

Other references · 21

  1. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,305
  2. 2023BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairscited by 520
  3. 2023PMC-VQA: Visual Instruction Tuning for Medical Visual Question Answeringcited by 338
  4. 2023Segment Anything Model for Medical Image Analysis: an Experimental Studycited by 291
  5. 2023Towards Generalist Biomedical AIcited by 289
  6. 2024Generalist Foundation Models from a Multimodal Dataset for 3D Computed Tomographycited by 199
  7. 2025Multimodal Chain-of-Thought Reasoning: A Comprehensive Surveycited by 182
  8. 2024M3D: Advancing 3D Medical Image Analysis with Multi-Modal Large Language Modelscited by 162
  9. 2025Hulu-Med: A Transparent Generalist Model towards Holistic Medical Vision-Language Understandingcited by 102
  10. 2024MedTrinity-25M: A Large-scale Multimodal Dataset with Multigranular Annotations for Medicinecited by 81
  11. 2024VISTA3D: A Unified Segmentation Foundation Model For 3D Medical Imagingcited by 64
  12. 2025Med3DVLM: An Efficient Vision-Language Model for 3D Medical Image Analysiscited by 46
9 more
  1. 2024RadGenome-Chest CT: A Grounded Vision-Language Dataset for Chest CT Analysiscited by 42
  2. 2024Towards a Holistic Framework for Multimodal Large Language Models in Three-dimensional Brain CT Report Generationcited by 31
  3. 2024Biomedical Visual Instruction Tuning with Clinician Preference Alignmentcited by 21
  4. 2024MC-CoT: A Modular Collaborative CoT Framework for Zero-shot Medical-VQA with LLM and MLLM Integrationcited by 21
  5. 2024Med-2E3: A 2D-Enhanced 3D Medical Multimodal Large Language Modelcited by 15
  6. 2025UMIT: Unifying Medical Imaging Tasks via Vision-Language Modelscited by 12
  7. 2026TerraScope: Pixel-Grounded Visual Reasoning for Earth Observationcited by 7
  8. 2026UniReason-Med: A Shared Grounded Reasoning Interface for 2D-to-3D Transfer in Medical VQAcited by 2
  9. 2026DyCo-RL: Dynamic Cross-Modal Coordination for Visual Reasoningcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Lost in Volume: The CT-SpatialVQA Benchmark for Evaluating Semantic-Spatial Understanding of 3D Medical Vision-Language Models
  2. 2026MedVL-SAM2: A unified 3D medical vision-language model for multimodal reasoning and prompt-driven segmentation
  3. 2026CORTEX: A Structured Reasoning Benchmark for Trustworthy 3D Chest CT MLLMs
  4. 2026ClinFusion: A Vision-Centric Multimodal LLM System for Holistic Medical Understanding
  5. 2026Volumetric Radiology AI in the Era of Multimodal Large Language Models
  6. 2025Enhancing 3D Medical Image Understanding with Pretraining Aided by 2D Multimodal Large Language Models
  7. 20253DReasonKnee: Advancing Grounded Reasoning in Medical Vision Language Models
  8. 2026Adapting 2D Multi-Modal Large Language Model for 3D CT Image Analysis