# MedDINOv3: How to adapt vision foundation models for medical image segmentation?

arXiv: https://arxiv.org/abs/2509.02379 · HTML: https://mlsanity.com/paper/2509.02379/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 48 references that are in the map.
- First posted on arXiv 2 Sep 2025.

## Builds on (most cited first, 30 of 34)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation](https://mlsanity.com/paper/2102.04306/) (2021)
- [Scaling Vision Transformers](https://mlsanity.com/paper/2106.04560/) (2021)
- [UNETR: Transformers for 3D Medical Image Segmentation](https://mlsanity.com/paper/2103.10504/) (2021)
- [Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images](https://mlsanity.com/paper/2201.01266/) (2022)
- [The Medical Segmentation Decathlon](https://mlsanity.com/paper/2106.05735/) (2021)
- [TotalSegmentator: robust segmentation of 104 anatomical structures in CT images](https://mlsanity.com/paper/2208.05868/) (2022)
- [The Liver Tumor Segmentation Benchmark (LiTS)](https://mlsanity.com/paper/1901.04056/) (2019)
- [A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPT](https://mlsanity.com/paper/2302.09419/) (2023)
- [AMOS: A Large-Scale Abdominal Multi-Organ Benchmark for Versatile Medical Image Segmentation](https://mlsanity.com/paper/2206.08023/) (2022)
- [CHAOS Challenge -- Combined (CT-MR) Healthy Abdominal Organ Segmentation](https://mlsanity.com/paper/2001.06535/) (2020)
- [nnU-Net: Self-adapting Framework for U-Net-Based Medical Image Segmentation](https://mlsanity.com/paper/1809.10486/) (2018)
- [nnU-Net Revisited: A Call for Rigorous Validation in 3D Medical Image Segmentation](https://mlsanity.com/paper/2404.09556/) (2024)
- [CoTr: Efficiently Bridging CNN and Transformer for 3D Medical Image Segmentation](https://mlsanity.com/paper/2103.03024/) (2021)
- [AbdomenCT-1K: Is Abdominal Organ Segmentation A Solved Problem?](https://mlsanity.com/paper/2010.14808/) (2020)
- [DeepOrgan: Multi-level Deep Convolutional Networks for Automated Pancreas Segmentation](https://mlsanity.com/paper/1506.06448/) (2015)
- [UTNet: A Hybrid Transformer Architecture for Medical Image Segmentation](https://mlsanity.com/paper/2107.00781/) (2021)
- [The KiTS21 Challenge: Automatic segmentation of kidneys, renal tumors, and renal cysts in corticomedullary-phase CT](https://mlsanity.com/paper/2307.01984/) (2023)
- [LeViT-UNet: Make Faster Encoders with Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2107.08623/) (2021)
- [Polyp-SAM: Transfer SAM for Polyp Segmentation](https://mlsanity.com/paper/2305.00293/) (2023)
- [Your ViT is Secretly an Image Segmentation Model](https://mlsanity.com/paper/2503.19108/) (2025)
- [UniMiSS: Universal Medical Self-Supervised Learning via Breaking Dimensionality Barrier](https://mlsanity.com/paper/2112.09356/) (2021)
- [RadGPT: Constructing 3D Image-Text Tumor Datasets](https://mlsanity.com/paper/2501.04678/) (2025)
- [Revisiting MAE pre-training for 3D medical image segmentation](https://mlsanity.com/paper/2410.23132/) (2024)
- [Evaluating General Purpose Vision Foundation Models for Medical Image Analysis: An Experimental Study of DINOv2 on Radiology Benchmarks](https://mlsanity.com/paper/2312.02366/) (2023)

## Built on by (most cited first, 25 of 25)

- [Does DINOv3 Set a New Medical Vision Standard? Benchmarking 2D and 3D Classification, Segmentation, and Registration](https://mlsanity.com/paper/2509.06467/) (2025)
- [BrainDINO: A Brain MRI Foundation Model for Generalizable Clinical Representation Learning](https://mlsanity.com/paper/2604.27277/) (2026)
- [Resolution scaling governs DINOv3 transfer performance in chest radiograph classification](https://mlsanity.com/paper/2510.07191/) (2025)
- [NeuroSeg Meets DINOv3: Transferring 2D Self-Supervised Visual Priors to 3D Neuron Segmentation via DINOv3 Initialization](https://mlsanity.com/paper/2603.23104/) (2026)
- [ChronosAD: Leveraging Time Series Foundation Models for Accurate Anomaly Detection](https://mlsanity.com/paper/2606.01300/) (2026)
- [Training-free Cross-domain Few-shot Segmentation via Robust Semantic Representation and Matching](https://mlsanity.com/paper/2606.24297/) (2026)
- [Improving Medical Image Generative Models with Fr'echet Distance Loss](https://mlsanity.com/paper/2607.13300/) (2026)
- [Vision4PPG: Emergent PPG Analysis Capability of Vision Foundation Models for Vital Signs like Blood Pressure](https://mlsanity.com/paper/2510.10366/) (2025)
- [SPIDER: Spatial Image CorresponDence Estimator for Robust Calibration](https://mlsanity.com/paper/2511.17750/) (2025)
- [Physically-Grounded Manifold Projection Model for Generalizable Metal Artifact Reduction in Dental CBCT](https://mlsanity.com/paper/2512.24260/) (2025)
- [Vascular anatomy-aware self-supervised pre-training for X-ray angiogram analysis](https://mlsanity.com/paper/2602.11536/) (2026)
- [Axial-Centric Cross-Plane Attention for 3D Medical Image Classification](https://mlsanity.com/paper/2602.21636/) (2026)
- [LUMOS: Latent Universal Medical Priors for Segmentation](https://mlsanity.com/paper/2603.01115/) (2026)
- [PlaneCycle: Training-Free 2D-to-3D Lifting of Foundation Models Without Adapters](https://mlsanity.com/paper/2603.04165/) (2026)
- [ConvNeXt-FD: A Fractal-Based Deep Model for Robust Biomedical Image Segmentation](https://mlsanity.com/paper/2605.22002/) (2026)
- [SegDINO: Introducing Multi-Scale Structure into DINO for Efficient Medical Image Segmentation](https://mlsanity.com/paper/2606.17972/) (2026)
- [Anatomically Consistent TMJ Disc Segmentation via Semantic Anchoring and Clinical Priors](https://mlsanity.com/paper/2606.21177/) (2026)
- [APRIL-MedSeg: A Modular Medical Image Segmentation Toolbox Embracing Modern Paradigms](https://mlsanity.com/paper/2606.30577/) (2026)
- [AnF-DiffPET: Anatomy- and Frequency-Guided Diffusion for PET/CT Denoising](https://mlsanity.com/paper/2607.00509/) (2026)
- [DynoDINO: Harnessing Dynamic Latent Information from DINO Features for Multi-Phase Medical Image Segmentation](https://mlsanity.com/paper/2607.29568/) (2026)
- [Lesion Detection in CT with Frozen Self-Distilled Features: SALT, a Spatially Adaptive Label-Guided Temperature](https://mlsanity.com/paper/2608.05100/) (2026)
- [DINO-3DRA: Leveraging 2D Foundation Model Semantics for 3D Cerebral Aneurysm Segmentation](https://mlsanity.com/paper/2608.07767/) (2026)
- [LoRCA: LoRA Cycle Adaptation for Histology to HiP-CT Translation with DINOv3](https://mlsanity.com/paper/2608.10002/) (2026)
- [Beyond Natural-Image Foundation Models: Benchmarking Satellite Pretraining for Ophthalmic Image Analysis](https://mlsanity.com/paper/2608.15195/) (2026)
- [CMRVision: A Foundation Model for Cardiac MR Image Analysis](https://mlsanity.com/paper/2609.01308/) (2026)

## Abstract (arXiv)

Accurate segmentation of organs and tumors in CT and MRI scans is essential for diagnosis, treatment planning, and disease monitoring. While deep learning has advanced automated segmentation, most models remain task-specific, lacking generalizability across modalities and institutions. Vision foundation models (FMs) pretrained on billion-scale natural images offer powerful and transferable representations. However, adapting them to medical imaging faces two key challenges: (1) the ViT backbone of most foundation models still underperform specialized CNNs on medical image segmentation, and (2) the large domain gap between natural and medical images limits transferability. We introduce MedDINOv3, a simple and effective framework for adapting DINOv3 to medical segmentation. We first revisit plain ViTs and design a simple and effective architecture with multi-scale token aggregation. Then, we perform domain-adaptive pretraining on CT-3M, a curated collection of 3.87M axial CT slices, using a multi-stage DINOv3 recipe to learn robust dense features. MedDINOv3 matches or exceeds state-of-the-art performance across four segmentation benchmarks, demonstrating the potential of vision foundation models as unified backbones for medical image segmentation. The code is available at https://github.com/ricklisz/MedDINOv3.