# General Purpose Image Encoder DINOv2 for Medical Image Registration

arXiv: https://arxiv.org/abs/2402.15687 · HTML: https://mlsanity.com/paper/2402.15687/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 25 references that are in the map.
- First posted on arXiv 24 Feb 2024.

## Builds on (most cited first, 15 of 15)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Vision Transformers Need Registers](https://mlsanity.com/paper/2309.16588/) (2023)
- [VoxelMorph: A Learning Framework for Deformable Medical Image Registration](https://mlsanity.com/paper/1809.05231/) (2018)
- [Large Deformation Diffeomorphic Image Registration with Laplacian Pyramid Networks](https://mlsanity.com/paper/2006.16148/) (2020)
- [Learn2Reg: comprehensive multi-task medical image registration challenge, dataset and evaluation in the era of deep learning](https://mlsanity.com/paper/2112.04489/) (2021)
- [Deep Learning in Medical Image Registration: A Survey](https://mlsanity.com/paper/1903.02026/) (2019)
- [Fast 3D registration with accurate optimisation and little learning for Learn2Reg 2021](https://mlsanity.com/paper/2112.03053/) (2021)
- [Evaluating General Purpose Vision Foundation Models for Medical Image Analysis: An Experimental Study of DINOv2 on Radiology Benchmarks](https://mlsanity.com/paper/2312.02366/) (2023)
- [Learning Deep Similarity Metric for 3D MR-TRUS Registration](https://mlsanity.com/paper/1806.04548/) (2018)
- [Cross-modal Attention for MRI and Ultrasound Volume Registration](https://mlsanity.com/paper/2107.04548/) (2021)
- [Voxelmorph++ Going beyond the cranial vault with keypoint supervision and multi-channel instance optimisation](https://mlsanity.com/paper/2203.00046/) (2022)
- [SAMConvex: Fast Discrete Optimization for CT Registration using Self-supervised Anatomical Embedding and Correlation Pyramid](https://mlsanity.com/paper/2307.09727/) (2023)

## Built on by (most cited first, 11 of 11)

- [EchoApex: A General-Purpose Vision Foundation Model for Echocardiography](https://mlsanity.com/paper/2410.11092/) (2024)
- [General Methods Make Great Domain-specific Foundation Models: A Case-study on Fetal Ultrasound](https://mlsanity.com/paper/2506.19552/) (2025)
- [MM-DINOv2: Adapting Foundation Models for Multi-Modal Medical Image Analysis](https://mlsanity.com/paper/2509.06617/) (2025)
- [Guiding Registration with Emergent Similarity from Pre-Trained Diffusion Models](https://mlsanity.com/paper/2506.02419/) (2025)
- [Graph Neural Networks: A suitable Alternative to MLPs in Latent 3D Medical Image Classification?](https://mlsanity.com/paper/2407.17219/) (2024)
- [Anonymization of Documents for Law Enforcement with Machine Learning](https://mlsanity.com/paper/2501.07334/) (2025)
- [General Vision Encoder Features as Guidance in Medical Image Registration](https://mlsanity.com/paper/2407.13311/) (2024)
- [DreamSat-2.0: Towards a General Single-View Asteroid 3D Reconstruction](https://mlsanity.com/paper/2508.01079/) (2025)
- [Foundation Models for Medical Imaging: Status, Challenges, and Directions](https://mlsanity.com/paper/2602.15913/) (2026)
- [Are Natural-Domain Foundation Models Effective for Accelerated Cardiac MRI Reconstruction?](https://mlsanity.com/paper/2604.22557/) (2026)
- [Cross-Modal Ultrasound-MRI Learning for Fetal Brain Ventricular Volumetry and Abnormality Screening](https://mlsanity.com/paper/2608.14763/) (2026)

## Abstract (arXiv)

Existing medical image registration algorithms rely on either dataset specific training or local texture-based features to align images. The former cannot be reliably implemented without large modality-specific training datasets, while the latter lacks global semantics thus could be easily trapped at local minima. In this paper, we present a training-free deformable image registration method, DINO-Reg, leveraging a general purpose image encoder DINOv2 for image feature extraction. The DINOv2 encoder was trained using the ImageNet data containing natural images. We used the pretrained DINOv2 without any finetuning. Our method feeds the DINOv2 encoded features into a discrete optimizer to find the optimal deformable registration field. We conducted a series of experiments to understand the behavior and role of such a general purpose image encoder in the application of image registration. Combined with handcrafted features, our method won the first place in the recent OncoReg Challenge. To our knowledge, this is the first application of general vision foundation models in medical image registration.