OmniCT: Towards a Unified Slice-Volume LVLM for Comprehensive CT Analysis
Not on the map. Why?
Abstract (arXiv)
Computed Tomography (CT) is one of the most widely used and diagnostically information-dense imaging modalities, covering critical organs such as the heart, lungs, liver, and colon. Clinical interpretation relies on both slice-driven local features (e.g., sub-centimeter nodules, lesion boundaries) and volume-driven spatial representations (e.g., tumor infiltration, inter-organ anatomical relations). However, existing Large Vision-Language Models (LVLMs) remain fragmented in CT slice versus volumetric understanding: slice-driven LVLMs show strong generalization but lack cross-slice spatial consistency, while volume-driven LVLMs explicitly capture volumetric semantics but suffer from coarse granularity and poor compatibility with slice inputs. The absence of a unified modeling paradigm constitutes a major bottleneck for the clinical translation of medical LVLMs. We present OmniCT, a powerful unified slice-volume LVLM for CT scenarios, which makes three contributions: (i) Spatial Consistency Enhancement (SCE): volumetric slice composition combined with tri-axial positional embedding that introduces volumetric consistency, and an MoE hybrid projection enables efficient slice-volume adaptation; (ii) Organ-level Semantic Enhancement (OSE): segmentation and ROI localization explicitly align anatomical regions, emphasizing lesion- and organ-level semantics; (iii) MedEval-CT: the largest slice-volume CT dataset and hybrid benchmark integrates comprehensive metrics for unified evaluation. OmniCT consistently outperforms existing methods with a substantial margin across diverse clinical tasks and satisfies both micro-level detail sensitivity and macro-level spatial reasoning. More importantly, it establishes a new paradigm for cross-modal medical imaging understanding. Our project is available at https://github.com/ZJU4HealthCare/OmniCT.
Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 57 of its 77 references that are in the map. First posted on arXiv 18 Feb 2026.
Builds on
57 of 77 references.
References · 57
- 2023GPT-4 Technical Reportcited by 18,132
- 2025Qwen3 Technical Reportcited by 8,195
- 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2024Qwen2 Technical Reportcited by 2,076
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
- 2025Gemma 3 Technical Reportcited by 1,686
- 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,385
38 more
- 2024Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysiscited by 1,375
- 2025DINOv3cited by 1,297
- 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,287
- 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,175
- 2022TotalSegmentator: robust segmentation of 104 anatomical structures in CT imagescited by 597
- 2023LLaMA-VID: An Image is Worth 2 Tokens in Large Language Modelscited by 539
- 2023VideoMAE V2: Scaling Video Masked Autoencoders with Dual Maskingcited by 516
- 2023BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairscited by 506
- 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understandingcited by 502
- 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 494
- 2025Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAscited by 491
- 2021SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answeringcited by 444
- 2025MedGemma Technical Reportcited by 398
- 2023Med-Flamingo: a Multimodal Medical Few-shot Learnercited by 387
- 2025HealthBench: Evaluating Large Language Models Towards Improved Human Healthcited by 385
- 2024PLLaVA : Parameter-free LLaVA Extension from Images to Videos for Video Dense Captioningcited by 320
- 2024Capabilities of Gemini Models in Medicinecited by 231
- 2025Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoningcited by 217
- 2023PMC-CLIP: Contrastive Language-Image Pre-training using Biomedical Documentscited by 212
- 2024HuatuoGPT-Vision, Towards Injecting Medical Visual Knowledge into Multimodal LLMs at Scalecited by 194
- 2024Generalist Foundation Models from a Multimodal Dataset for 3D Computed Tomographycited by 192
- 2024OmniMedVQA: A New Large-Scale Comprehensive Evaluation Benchmark for Medical LVLMcited by 182
- 2025MiniCPM-V 4.5: Cooking Efficient MLLMs via Architecture, Data, and Training Recipecited by 174
- 2025MedVLM-R1: Incentivizing Medical Reasoning Capability of Vision-Language Models (VLMs) via Reinforcement Learningcited by 172
- 2024M3D: Advancing 3D Medical Image Analysis with Multi-Modal Large Language Modelscited by 154
- 2024LITA: Language Instructed Temporal-Localization Assistantcited by 124
- 2025HealthGPT: A Medical Large Vision-Language Model for Unifying Comprehension and Generation via Heterogeneous Knowledge Adaptationcited by 112
- 2023BioMedGPT: Open Multimodal Generative Pre-trained Transformer for BioMedicinecited by 99
- 2024CT2Rep: Automated Radiology Report Generation for 3D Medical Imagingcited by 96
- 2024VideoRefer Suite: Advancing Spatial-Temporal Object Understanding with Video LLMcited by 71
- 2025Large-scale and Fine-grained Vision-language Pre-training for Enhanced CT Image Understandingcited by 53
- 2024Med-MoE: Mixture of Domain-Specific Experts for Lightweight Medical Vision-Language Modelscited by 43
- 2023CXR-LLAVA: a multimodal large language model for interpreting chest X-ray imagescited by 37
- 2025Capabilities of GPT-5 on Multimodal Medical Reasoningcited by 35
- 2025RadGPT: Constructing 3D Image-Text Tumor Datasetscited by 34
- 2023PSVT: End-to-End Multi-person 3D Pose and Shape Estimation with Progressive Video Transformerscited by 31
- 2022Learning Spatiotemporal Frequency-Transformer for Compressed Video Super-Resolutioncited by 25
- 20253D-RAD: A Comprehensive 3D Radiology Med-VQA Dataset with Multi-Temporal Analysis and Diverse Diagnostic Taskscited by 24
50 most cited of 57 All 57 (JSON)
Built on by
Citing papers per year: 2026: 9.
9 papers.
- 2026InstructSAM: Segment Any Instance with Any Instructionscited by 6
- 2026Regulating Anatomy-Aware Rewards via Trajectory-Integral Feedback for Volumetric Computed Tomography Analysiscited by 2
- 2026CT-Flow: Orchestrating CT Interpretation Workflow with Model Context Protocol Serverscited by 1
- 2026RadSight: Towards Perceptually Reliable Multimodal Radiology Image Understandingcited by 1
- 2026HounsWorld: A Multimodal World Model for Hidden Patient-State Readout, Reconstruction, and Simulationcited by 0
- 2026Holtercare-Bench: A Multimodal Benchmark for Evaluating Long-Term Dynamic ECG Analysiscited by 0
- 2026Volumetric Radiology AI in the Era of Multimodal Large Language Modelscited by 0
- 2026SeVeR: Selective Visual Exposure and Retrieval for 3D Medical Image Question Answeringcited by 0
- 2026SelfCue: Making a 3D CT Report Generator Say What It Already Knowscited by 0
- 2026SelfCue: Making a 3D CT Report Generator Say What It Already Knows25 Sep 2026
- 2026SeVeR: Selective Visual Exposure and Retrieval for 3D Medical Image Question Answering26 Aug 2026
- 2026Volumetric Radiology AI in the Era of Multimodal Large Language Models20 Aug 2026
- 2026Holtercare-Bench: A Multimodal Benchmark for Evaluating Long-Term Dynamic ECG Analysis19 Aug 2026
- 2026HounsWorld: A Multimodal World Model for Hidden Patient-State Readout, Reconstruction, and Simulation13 Aug 2026
- 2026RadSight: Towards Perceptually Reliable Multimodal Radiology Image Understanding24 Jul 2026
- 2026InstructSAM: Segment Any Instance with Any Instructions25 May 2026
- 2026Regulating Anatomy-Aware Rewards via Trajectory-Integral Feedback for Volumetric Computed Tomography Analysis19 May 2026
- 2026CT-Flow: Orchestrating CT Interpretation Workflow with Model Context Protocol Servers23 Feb 2026
Similar, not cited
Close in wording, no citation either way
- 2024Read Like a Radiologist: Efficient Vision-Language Model for 3D Medical Imaging Interpretation
- 2025Unified Supervision For Vision-Language Modeling in 3D Computed Tomography
- 2025MOSAIC: A Multi-View 2.5D Organ Slice Selector with Cross-Attentional Reasoning for Anatomically-Aware CT Localization in Medical Organ Segmentation
- 2025From Alignment to Synthesis: Contrastive Volumetric Grounding for Text-to-CT Generation
- 2025SCALE-VLP: Soft-Weighted Contrastive Volumetric Vision-Language Pre-training with Spatial-Knowledge Semantics
- 2026CARVE: Cross-Slice Anisotropic Reallocation of Visual Evidence for Efficient 3D Medical Volume Understanding
- 2026Towards Enhancing 3D Spatial Reasoning in Medical Multimodal Large Language Models
- 2025VELVET-Med: Vision and Efficient Language Pre-training for Volumetric Imaging Tasks in Medicine