mlsanityData to 14 Oct 2026

SCALE-VLP: Soft-Weighted Contrastive Volumetric Vision-Language Pre-training with Spatial-Knowledge Semantics

arXiv 2511.02996 ↗First posted 4 Nov 2025cs.CVcited by 2

On the map: Multimodal LLMs / vision-language models → Medical vision-language models

Abstract (arXiv)

Vision-language models (VLMs) have demonstrated strong cross-modal capabilities, yet most work remains limited to 2D data and assumes binary supervision (i.e., positive vs. negative pairs), overlooking the continuous and structured dependencies present in volumetric data such as CT. Existing approaches often treat volumetric scans as independent 2D slices, compromising spatial coherence and underutilizing rich clinical semantics. We propose SCALE-VLP, a soft-weighted contrastive vision-language pre-training framework that integrates (i) volumetric spatial semantics to preserve anatomical structure and (ii) domain-aware, knowledge-infused semantics (e.g., radiological ontologies) to guide alignment. This yields structurally consistent and semantically grounded representations under limited supervision, demonstrating strong cross-task transferability (retrieval, report generation, and classification), and cross-domain generalizability with consistent gains without further fine-tuning. In particular, compared to the previous state of the art, SCALE-VLP achieves up to 4.3x higher top-1 CT-report retrieval, improves abnormality classification by 10 points, and reaches ROUGE-L 0.44 and BERT-F1 0.89 for report generation. Further, in zero-shot evaluation on an out-of-domain external dataset, we observe consistent gains, indicating the cross-task and cross-domain generalization ability of SCALE-VLP.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 11 of its 37 references that are in the map. First posted on arXiv 4 Nov 2025.

Builds on

11 of 37 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Large-scale and Fine-grained Vision-language Pre-training for Enhanced CT Image Understanding24 Jan
  2. 2024M3D: Advancing 3D Medical Image Analysis with Multi-Modal Large Language Models31 Mar
  3. 2023T3D: Advancing 3D Medical Vision-Language Pre-training by Learning Multi-View Visual Consistency3 Dec

Other references · 8

  1. 2019Publicly Available Clinical BERT Embeddingscited by 1,002
  2. 2024BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domainscited by 301
  3. 2023MEDITRON-70B: Scaling Medical Pretraining for Large Language Modelscited by 274
  4. 2024HuatuoGPT-o1, Towards Medical Complex Reasoning with LLMscited by 265
  5. 2024Generalist Foundation Models from a Multimodal Dataset for 3D Computed Tomographycited by 199
  6. 2020Show, Describe and Conclude: On Exploiting the Structure Information of Chest X-Ray Reportscited by 58
  7. 2024CT-GLIP: 3D Grounded Language-Image Pretraining with CT Scans and Radiology Reports for Full-Body Scenarioscited by 30
  8. 2025How Well Do Supervised 3D Models Transfer to Medical Imaging Tasks?cited by 19

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Curriculum-Driven 3D CT Report Generation via Language-Free Visual Grafting and Zone-Constrained Compressioncited by 3
  2. 2026Volumetric Radiology AI in the Era of Multimodal Large Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025More performant and scalable: Rethinking contrastive vision-language pre-training of radiology in the LLM era
  2. 2026Learning to Read Where to Look: Disease-Aware Vision-Language Pretraining for 3D CT
  3. 2025VELVET-Med: Vision and Efficient Language Pre-training for Volumetric Imaging Tasks in Medicine
  4. 2025Boosting Vision Semantic Density with Anatomy Normality Modeling for Medical Vision-language Pre-training
  5. 2024MG-3D: Multi-Grained Knowledge-Enhanced 3D Medical Vision-Language Pre-training
  6. 2026Disease-Centric Vision-Language Pretraining with Hybrid Visual Encoding for 3D Computed Tomography
  7. 2026ASAP: Advancing Medical Volumetric Representation Learning with Anatomy-aware Semantically-adaptive Pre-training
  8. 2025Scaling Down to Scale Up: Towards Operationally-Efficient and Deployable Clinical Models via Cross-Modal Low-Rank Adaptation for Medical Vision-Language Models