# Evaluating General Purpose Vision Foundation Models for Medical Image Analysis: An Experimental Study of DINOv2 on Radiology Benchmarks

arXiv: https://arxiv.org/abs/2312.02366 · HTML: https://mlsanity.com/paper/2312.02366/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 48 references that are in the map.
- First posted on arXiv 4 Dec 2023.

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison](https://mlsanity.com/paper/1901.07031/) (2019)
- [BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models](https://mlsanity.com/paper/2106.10199/) (2021)
- [ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases](https://mlsanity.com/paper/1705.02315/) (2017)
- [Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images](https://mlsanity.com/paper/2201.01266/) (2022)
- [Florence: A New Foundation Model for Computer Vision](https://mlsanity.com/paper/2111.11432/) (2021)
- [CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning](https://mlsanity.com/paper/1711.05225/) (2017)
- [Self-Supervised Pre-Training of Swin Transformers for 3D Medical Image Analysis](https://mlsanity.com/paper/2111.14791/) (2021)
- [Big Self-Supervised Models Advance Medical Image Classification](https://mlsanity.com/paper/2101.05224/) (2021)
- [Google Landmarks Dataset v2 -- A Large-Scale Benchmark for Instance-Level Recognition and Retrieval](https://mlsanity.com/paper/2004.01804/) (2020)
- [Segment Anything Model for Medical Image Analysis: an Experimental Study](https://mlsanity.com/paper/2304.10517/) (2023)
- [Adapting Pretrained Vision-Language Foundational Models to Medical Imaging Domains](https://mlsanity.com/paper/2210.04133/) (2022)
- [Foundation Model for Endoscopy Video Analysis via Large-scale Self-supervised Pre-train](https://mlsanity.com/paper/2306.16741/) (2023)
- [On the limits of cross-domain generalization in automated X-ray prediction](https://mlsanity.com/paper/2002.02497/) (2020)
- [Parameter-Efficient Fine-Tuning for Medical Image Analysis: The Missed Opportunity](https://mlsanity.com/paper/2305.08252/) (2023)
- [Computer-Vision Benchmark Segment-Anything Model (SAM) in Medical Images: Accuracy in 12 Datasets](https://mlsanity.com/paper/2304.09324/) (2023)
- [Supervised Transfer Learning at Scale for Medical Imaging](https://mlsanity.com/paper/2101.05913/) (2021)
- [VisionFM: a Multi-Modal Multi-Task Vision Foundation Model for Generalist Ophthalmic Artificial Intelligence](https://mlsanity.com/paper/2310.04992/) (2023)
- [Towards Segment Anything Model (SAM) for Medical Image Segmentation: A Survey](https://mlsanity.com/paper/2305.03678/) (2023)
- [Disruptive Autoencoders: Leveraging Low-level features for 3D Medical Image Pre-training](https://mlsanity.com/paper/2307.16896/) (2023)
- [Learning Optimal K-space Acquisition and Reconstruction using Physics-Informed Neural Networks](https://mlsanity.com/paper/2204.02480/) (2022)

## Built on by (most cited first, 30 of 33)

- [MedDINOv3: How to adapt vision foundation models for medical image segmentation?](https://mlsanity.com/paper/2509.02379/) (2025)
- [Comparative Analysis of ImageNet Pre-Trained Deep Learning Models and DINOv2 in Medical Imaging Classification](https://mlsanity.com/paper/2402.07595/) (2024)
- [General Purpose Image Encoder DINOv2 for Medical Image Registration](https://mlsanity.com/paper/2402.15687/) (2024)
- [Towards Generalisable Foundation Models for Brain MRI](https://mlsanity.com/paper/2510.23415/) (2025)
- [Revisiting 2D Foundation Models for Scalable 3D Medical Image Classification](https://mlsanity.com/paper/2512.12887/) (2025)
- [MM-DINOv2: Adapting Foundation Models for Multi-Modal Medical Image Analysis](https://mlsanity.com/paper/2509.06617/) (2025)
- [SciVid: Cross-Domain Evaluation of Video Models in Scientific Applications](https://mlsanity.com/paper/2507.03578/) (2025)
- [Graph Neural Networks: A suitable Alternative to MLPs in Latent 3D Medical Image Classification?](https://mlsanity.com/paper/2407.17219/) (2024)
- [FastDINOv2: Frequency Based Curriculum Learning Improves Robustness and Training Speed](https://mlsanity.com/paper/2507.03779/) (2025)
- [DinoAtten3D: Slice-Level Attention Aggregation of DinoV2 for 3D Brain MRI Anomaly Classification](https://mlsanity.com/paper/2509.12512/) (2025)
- [TAP-CT: 3D Task-Agnostic Pretraining of Computed Tomography Foundation Models](https://mlsanity.com/paper/2512.00872/) (2025)
- [VQ-Seg: Vector-Quantized Token Perturbation for Semi-Supervised Medical Image Segmentation](https://mlsanity.com/paper/2601.10124/) (2026)
- [Brewing Stronger Features: Dual-Teacher Distillation for Multispectral Earth Observation](https://mlsanity.com/paper/2602.19863/) (2026)
- [DINO-LG: Enhancing Vision Transformers with Label Guidance for Coronary Artery Calcium Detection](https://mlsanity.com/paper/2411.07976/) (2024)
- [VFM-Guided Semi-Supervised Detection Transformer under Source-Free Constraints for Remote Sensing Object Detection](https://mlsanity.com/paper/2508.11167/) (2025)
- [Multi Anatomy X-Ray Foundation Model](https://mlsanity.com/paper/2509.12146/) (2025)
- [TomoGraphView: 3D Medical Image Classification with Omnidirectional Slice Representations and Graph Neural Networks](https://mlsanity.com/paper/2511.09605/) (2025)
- [Self-supervised Pretraining of Cell Segmentation Models](https://mlsanity.com/paper/2604.10609/) (2026)
- [When Do Cheap Probes Predict Expensive Training? Probing 3D-CT Encoders for Text Generation](https://mlsanity.com/paper/2607.22771/) (2026)
- [Foundation Model-Enabled Efficient Data Sampling (FEEDS): A label-efficient training strategy for pan-cancer, multi-tracer PET/CT datasets](https://mlsanity.com/paper/2608.11076/) (2026)
- [VESSA: Video-based objEct-centric Self-Supervised Adaptation for Visual Foundation Models](https://mlsanity.com/paper/2510.20994/) (2025)
- [DBT-DINO: Towards Foundation model based analysis of Digital Breast Tomosynthesis](https://mlsanity.com/paper/2512.13608/) (2025)
- [Axial-Centric Cross-Plane Attention for 3D Medical Image Classification](https://mlsanity.com/paper/2602.21636/) (2026)
- [Are Natural-Domain Foundation Models Effective for Accelerated Cardiac MRI Reconstruction?](https://mlsanity.com/paper/2604.22557/) (2026)
- [Capability neq Interpretability: Human Interpretability of Vision Foundation Models](https://mlsanity.com/paper/2605.20337/) (2026)
- [DALE-CT: Depth-Aware 2D Slice Encoders Learn an Anatomical World Model of Chest CT](https://mlsanity.com/paper/2606.07775/) (2026)
- [Recover Semantics First, Generate Better: Improved Latent Modeling for 3D MRI Reconstruction and Cross-Contrast Synthesis](https://mlsanity.com/paper/2606.17989/) (2026)
- [In-Context Learning for Wound Classification with Small Multimodal Language Models](https://mlsanity.com/paper/2607.18819/) (2026)
- [LoRCA: LoRA Cycle Adaptation for Histology to HiP-CT Translation with DINOv3](https://mlsanity.com/paper/2608.10002/) (2026)
- [From Interpretability Methods to Interpretable Models](https://mlsanity.com/paper/2609.05399/) (2026)

## Abstract (arXiv)

The integration of deep learning systems into healthcare has been hindered by the resource-intensive process of data annotation and the inability of these systems to generalize to different data distributions. Foundation models, which are models pre-trained on large datasets, have emerged as a solution to reduce reliance on annotated data and enhance model generalizability and robustness. DINOv2 is an open-source foundation model pre-trained with self-supervised learning on 142 million curated natural images that exhibits promising capabilities across various vision tasks. Nevertheless, a critical question remains unanswered regarding DINOv2's adaptability to radiological imaging, and whether its features are sufficiently general to benefit radiology image analysis. Therefore, this study comprehensively evaluates the performance DINOv2 for radiology, conducting over 200 evaluations across diverse modalities (X-ray, CT, and MRI). To measure the effectiveness and generalizability of DINOv2's feature representations, we analyze the model across medical image analysis tasks including disease classification and organ segmentation on both 2D and 3D images, and under different settings like kNN, few-shot learning, linear-probing, end-to-end fine-tuning, and parameter-efficient fine-tuning. Comparative analyses with established supervised, self-supervised, and weakly-supervised models reveal DINOv2's superior performance and cross-task generalizability. The findings contribute insights to potential avenues for optimizing pre-training strategies for medical imaging and enhancing the broader understanding of DINOv2's role in bridging the gap between natural and radiological image analysis. Our code is available at https://github.com/MohammedSB/DINOv2ForRadiology