GreenRFM: Learning a resource-efficient radiology vision-language foundation model via supervision-centric pre-training
On the map: Multimodal LLMs / vision-language models → Medical vision-language models
Abstract (arXiv)
Radiology foundation models (RFMs) have largely inherited the scale-first recipe of natural-image vision--language pre-training. This recipe is difficult to deploy in 3D radiology, where training corpora are smaller, reports vary across institutions, and receiving hospitals often need local adaptation under privacy and compute constraints. We ask whether routine radiology reports can instead be converted into auditable diagnostic supervision that shapes the image encoder, text encoder, aligned space, and local-adaptation procedure. We develop GreenRFM, a supervision-centric pre-training framework organized around four empirical principles: More distilled, Ubiquitous, Semantic-enforcing, and Task-aligning (MUST) supervision. These principles convert noisy reports into structured diagnostic signals and use them to learn discriminative unimodal encoders plus an aligned image--text space for diagnosis-centered multimodal use. GreenRFM requires 24 GPU-hours on a single 24GB GPU (lightweight variant: 6GB VRAM, 4~hours) and reaches a zero-shot CT-RATE AUC of 84.8. Evaluations using more than 200,000 volumes from six institutions and two modalities show transfer to private clinical cohorts and to musculoskeletal MRI. On a local institutional cohort, computationally feasible retraining raises macro-AUC from 70.5 to 82.1. The aligned space also improves hepatocellular-carcinoma microvascular-invasion prediction and trans-arterial chemoembolization response analysis over established clinical scores. These results support supervision-centric pre-training as a practical route to resource-efficient, locally adaptable, diagnosis-centered radiology vision--language representations.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 37 of its 67 references that are in the map. First posted on arXiv 6 Mar 2026.
Builds on
37 of 67 references.
Closest ancestors · 7
Same direction, newest first
- 2025More performant and scalable: Rethinking contrastive vision-language pre-training of radiology in the LLM era16 Sep
- 2025SimCroP: Radiograph Representation Learning with Similarity-driven Cross-granularity Pre-training10 Sep
- 2025Unified Supervision For Vision-Language Modeling in 3D Computed Tomography1 Sep
- 2025Boosting Vision Semantic Density with Anatomy Normality Modeling for Medical Vision-language Pre-training1 Aug
- 2025A Reality Check of Vision-Language Pre-training in Radiology: Have We Progressed Using Text?7 Apr
- 2025DCFormer: Efficient 3D Vision-Language Modeling with Decomposed Convolutions7 Feb
- 2025Large-scale and Fine-grained Vision-language Pre-training for Enhanced CT Image Understanding24 Jan
Other references · 30
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
- 2023Visual Instruction Tuningcited by 7,792
- 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
- 2017On Calibration of Modern Neural Networkscited by 4,860
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
- 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
- 2023Qwen Technical Reportcited by 3,273
- 2017The Kinetics Human Action Video Datasetcited by 1,894
- 2019Similarity of Neural Network Representations Revisitedcited by 1,695
- 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,410
18 more
- 2020Contrastive Learning of Medical Visual Representations from Paired Images and Textcited by 636
- 2020The Future of Digital Health with Federated Learningcited by 625
- 2022TotalSegmentator: robust segmentation of 104 anatomical structures in CT imagescited by 597
- 2017A systematic study of the class imbalance problem in convolutional neural networkscited by 510
- 2022Making the Most of Text Semantics to Improve Biomedical Vision--Language Processingcited by 336
- 2021MagFace: A Universal Representation for Face Recognition and Quality Assessmentcited by 332
- 2017NormFace: L2 Hypersphere Embedding for Face Verificationcited by 237
- 2017Deep Learning is Robust to Massive Label Noisecited by 164
- 2020Machine-Learning-Based Multiple Abnormality Prediction with Large-Scale Chest Computed Tomography Volumescited by 66
- 2024Large-Scale 3D Medical Image Pre-training with Geometric Context Priorscited by 34
- 2024Large Language Model with Region-guided Referring and Grounding for CT Report Generationcited by 27
- 2024Foundation Models in Radiology: What, How, When, Why and Why Notcited by 26
- 2024Bootstrapping Chest CT Image Understanding by Distilling Knowledge from X-ray Expert Modelscited by 24
- 2025Pillar-0: A New Frontier for Radiology Foundation Modelscited by 24
- 2025Towards Scalable Language-Image Pre-training for 3D Medical Imagingcited by 13
- 2025Bridged Semantic Alignment for Zero-shot 3D Medical Image Diagnosiscited by 9
- 2025LEAVS: An LLM-based Labeler for Abdominal CT Supervisioncited by 3
- 2025X2CT-CLIP: Enable Multi-Abnormality Detection in Computed Tomography from Chest Radiography via Tri-Modal Contrastive Learningcited by 2
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026RadPRISM: Schema-stratified radiology-report supervision for concept-disentangled image representations and visual grounding
- 2025Comprehensive language-image pre-training for 3D medical image understanding
- 2026U-VLM: Hierarchical Vision Language Modeling for Report Generation
- 2024Argus: Benchmarking and Enhancing Vision-Language Models for 3D Radiology Report Generation
- 2026RadJEPA: Radiology Encoder for Chest X-Rays via Joint Embedding Predictive Architecture
- 2026Learning to Read Where to Look: Disease-Aware Vision-Language Pretraining for 3D CT
- 2023Vision-Language Modelling For Radiological Imaging and Reports In The Low Data Regime
- 2026Scalable Training of Spatially Grounded 2D Vision-Language Models for Radiology