# GreenRFM: Learning a resource-efficient radiology vision-language foundation model via supervision-centric pre-training

arXiv: https://arxiv.org/abs/2603.06467 · HTML: https://mlsanity.com/paper/2603.06467/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 37 of its 67 references that are in the map.
- First posted on arXiv 6 Mar 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Medical vision-language models](https://mlsanity.com/mllm/medical-vision-language-models/)

## Builds on (most cited first, 30 of 37)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Flamingo: a Visual Language Model for Few-Shot Learning](https://mlsanity.com/paper/2204.14198/) (2022)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Qwen Technical Report](https://mlsanity.com/paper/2309.16609/) (2023)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison](https://mlsanity.com/paper/1901.07031/) (2019)
- [Contrastive Learning of Medical Visual Representations from Paired Images and Text](https://mlsanity.com/paper/2010.00747/) (2020)
- [The Future of Digital Health with Federated Learning](https://mlsanity.com/paper/2003.08119/) (2020)
- [TotalSegmentator: robust segmentation of 104 anatomical structures in CT images](https://mlsanity.com/paper/2208.05868/) (2022)
- [A systematic study of the class imbalance problem in convolutional neural networks](https://mlsanity.com/paper/1710.05381/) (2017)
- [Making the Most of Text Semantics to Improve Biomedical Vision--Language Processing](https://mlsanity.com/paper/2204.09817/) (2022)
- [MagFace: A Universal Representation for Face Recognition and Quality Assessment](https://mlsanity.com/paper/2103.06627/) (2021)
- [NormFace: L2 Hypersphere Embedding for Face Verification](https://mlsanity.com/paper/1704.06369/) (2017)
- [Deep Learning is Robust to Massive Label Noise](https://mlsanity.com/paper/1705.10694/) (2017)
- [Machine-Learning-Based Multiple Abnormality Prediction with Large-Scale Chest Computed Tomography Volumes](https://mlsanity.com/paper/2002.04752/) (2020)
- [Large-scale and Fine-grained Vision-language Pre-training for Enhanced CT Image Understanding](https://mlsanity.com/paper/2501.14548/) (2025)
- [Large-Scale 3D Medical Image Pre-training with Geometric Context Priors](https://mlsanity.com/paper/2410.09890/) (2024)
- [Large Language Model with Region-guided Referring and Grounding for CT Report Generation](https://mlsanity.com/paper/2411.15539/) (2024)
- [Foundation Models in Radiology: What, How, When, Why and Why Not](https://mlsanity.com/paper/2411.18730/) (2024)
- [Pillar-0: A New Frontier for Radiology Foundation Models](https://mlsanity.com/paper/2511.17803/) (2025)
- [Bootstrapping Chest CT Image Understanding by Distilling Knowledge from X-ray Expert Models](https://mlsanity.com/paper/2404.04936/) (2024)
- [Towards Scalable Language-Image Pre-training for 3D Medical Imaging](https://mlsanity.com/paper/2505.21862/) (2025)
- [Boosting Vision Semantic Density with Anatomy Normality Modeling for Medical Vision-language Pre-training](https://mlsanity.com/paper/2508.03742/) (2025)
- [DCFormer: Efficient 3D Vision-Language Modeling with Decomposed Convolutions](https://mlsanity.com/paper/2502.05091/) (2025)

## Built on by (most cited first, 2 of 2)

- [ORACLE-CT: Anatomy-Aware Support Pooling for CT Classification](https://mlsanity.com/paper/2606.05460/) (2026)
- [ARC-CT: Anatomy-Routed Contrastive Vision-Language Learning for 3D Chest CT](https://mlsanity.com/paper/2608.28455/) (2026)

## Abstract (arXiv)

Radiology foundation models (RFMs) have largely inherited the scale-first recipe of natural-image vision--language pre-training. This recipe is difficult to deploy in 3D radiology, where training corpora are smaller, reports vary across institutions, and receiving hospitals often need local adaptation under privacy and compute constraints. We ask whether routine radiology reports can instead be converted into auditable diagnostic supervision that shapes the image encoder, text encoder, aligned space, and local-adaptation procedure. We develop GreenRFM, a supervision-centric pre-training framework organized around four empirical principles: More distilled, Ubiquitous, Semantic-enforcing, and Task-aligning (MUST) supervision. These principles convert noisy reports into structured diagnostic signals and use them to learn discriminative unimodal encoders plus an aligned image--text space for diagnosis-centered multimodal use. GreenRFM requires 24 GPU-hours on a single 24GB GPU (lightweight variant: 6GB VRAM, 4~hours) and reaches a zero-shot CT-RATE AUC of 84.8. Evaluations using more than 200,000 volumes from six institutions and two modalities show transfer to private clinical cohorts and to musculoskeletal MRI. On a local institutional cohort, computationally feasible retraining raises macro-AUC from 70.5 to 82.1. The aligned space also improves hepatocellular-carcinoma microvascular-invasion prediction and trans-arterial chemoembolization response analysis over established clinical scores. These results support supervision-centric pre-training as a practical route to resource-efficient, locally adaptable, diagnosis-centered radiology vision--language representations.