TumorChain: Interleaved Multimodal Chain-of-Thought Reasoning for Traceable Clinical Tumor Analysis
On the map: LLM reasoning → Clinical and biomedical applications
Abstract (arXiv)
Accurate tumor analysis is central to clinical radiology and precision oncology, where early detection, reliable lesion characterization, and pathology-level risk assessment guide diagnosis and treatment planning. Chain-of-Thought (CoT) reasoning is particularly important in this setting because it enables step-by-step interpretation from imaging findings to clinical impressions and pathology conclusions, improving traceability and reducing diagnostic errors. Here, we target the clinical tumor analysis task and build a large-scale benchmark that operationalizes a multimodal reasoning pipeline, spanning findings, impressions, and pathology predictions. We curate TumorCoT, a large-scale dataset of 1.5M CoT-labeled VQA instructions paired with 3D CT scans, with step-aligned rationales and cross-modal alignments along the trajectory from findings to impression to pathology, enabling evaluation of both answer accuracy and reasoning consistency. We further propose TumorChain, a multimodal interleaved reasoning framework that tightly couples 3D imaging encoders, clinical text understanding, and organ-level vision-language alignment. Through cross-modal alignment and iterative interleaved causal reasoning, TumorChain grounds visual evidence, aggregates conclusions, and issues pathology predictions after multiple rounds of self-refinement, improving traceability and reducing hallucination risk. Experiments show consistent improvements over strong baselines in lesion detection, impression generation, and pathology classification, and demonstrate strong generalization on the DeepTumorVQA benchmark. These results highlight the potential of multimodal reasoning for reliable and interpretable tumor analysis in clinical practice. Detailed information about our project can be found on our project homepage at https://github.com/ZJU4HealthCare/TumorChain.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning. Builds on 24 of its 49 references that are in the map. First posted on arXiv 6 Mar 2026.
Builds on
24 of 49 references.
Closest ancestors · 7
Same direction, newest first
- 2025MedVLThinker: Simple Baselines for Multimodal Medical Reasoning4 Aug
- 2025CX-Mind: A Pioneering Multimodal Large Language Model for Interleaved Reasoning in Chest X-ray via Curriculum-Guided Reinforcement Learning31 Jul
- 2025ReasonMed: A 370K Multi-Agent Generated Dataset for Advancing Medical Reasoning11 Jun
- 2025Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoning8 Jun
- 2025Are Vision Language Models Ready for Clinical Diagnosis? A 3D Medical Benchmark for Tumor-centric Visual Question Answering25 May
- 2025Med-R1: Reinforcement Learning for Generalizable Medical Reasoning in Vision-Language Models18 Mar
- 2025MedVLM-R1: Incentivizing Medical Reasoning Capability of Vision-Language Models (VLMs) via Reinforcement Learning26 Feb
Other references · 17
- 2023GPT-4 Technical Reportcited by 18,396
- 2025Qwen3 Technical Reportcited by 8,820
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
- 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 923
- 2020PathVQA: 30000+ Questions for Medical Visual Question Answeringcited by 475
- 2025HealthBench: Evaluating Large Language Models Towards Improved Human Healthcited by 429
- 2024Generalist Foundation Models from a Multimodal Dataset for 3D Computed Tomographycited by 199
- 2024M3D: Advancing 3D Medical Image Analysis with Multi-Modal Large Language Modelscited by 162
- 2025Adaptive Chain-of-Focus Reasoning via Dynamic Visual Search and Zooming for Efficient VLMscited by 96
- 2024Advancing Multimodal Medical Capabilities of Geminicited by 90
- 2024RaTEScore: A Metric for Radiology Report Generationcited by 65
5 more
- 2025Med3DVLM: An Efficient Vision-Language Model for 3D Medical Image Analysiscited by 46
- 20253D-RAD: A Comprehensive 3D Radiology Med-VQA Dataset with Multi-Temporal Analysis and Diverse Diagnostic Taskscited by 26
- 2025EyecareGPT: Boosting Comprehensive Ophthalmology Understanding with Tailored Dataset, Benchmark and Modelcited by 22
- 2025MedResearcher-R1: Expert-Level Medical Deep Researcher via A Knowledge-Informed Trajectory Synthesis Frameworkcited by 22
- 2025Interleaved Reasoning for Large Language Models via Reinforcement Learningcited by 21
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026Regulating Anatomy-Aware Rewards via Trajectory-Integral Feedback for Volumetric Computed Tomography Analysiscited by 2
- 2026OncoTriad-QA: A Patient-Level Radiology-Pathology-Genomics Benchmark for Pan-Cancer Reasoningcited by 0
- 2026EndoVLM: An Endoscopy Vision-Language Pre-training Model via Anatomy-Guided Sparsity and Progressive Alignmentcited by 0
- 2026EndoVLM: An Endoscopy Vision-Language Pre-training Model via Anatomy-Guided Sparsity and Progressive Alignment5 Aug 2026
- 2026OncoTriad-QA: A Patient-Level Radiology-Pathology-Genomics Benchmark for Pan-Cancer Reasoning21 May 2026
- 2026Regulating Anatomy-Aware Rewards via Trajectory-Integral Feedback for Volumetric Computed Tomography Analysis19 May 2026
Similar, not cited
Close in wording, no citation either way
- 2026M3CoTBench: Benchmark Chain-of-Thought of MLLMs in Medical Image Understanding
- 2026CORTEX: A Structured Reasoning Benchmark for Trustworthy 3D Chest CT MLLMs
- 2026DeepTumorVQA: A Hierarchical 3D CT Benchmark for Stage-Wise Evaluation of Medical VLMs and Tool-Augmented Agents
- 2026Automated Report-Derived Oncology VQA Benchmark for Evaluating Vision-Language Models on 3D Medical Imaging
- 2026BreastGPT: A Multimodal Large Language Model for the Full Spectrum of Breast Cancer Clinical Routine
- 2025Adaptive Diagnostic Reasoning Framework for Pathology with Multimodal Large Language Models
- 2025A Versatile Pathology Co-pilot via Reasoning Enhanced Multimodal Large Language Model
- 2025PathoHR: Hierarchical Reasoning for Vision-Language Models in Pathology