# SAMReg: SAM-enabled Image Registration with ROI-based Correspondence

arXiv: https://arxiv.org/abs/2410.14083 · HTML: https://mlsanity.com/paper/2410.14083/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 62 references that are in the map.
- First posted on arXiv 17 Oct 2024.

## Builds on (most cited first, 28 of 28)

- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [VoxelMorph: A Learning Framework for Deformable Medical Image Registration](https://mlsanity.com/paper/1809.05231/) (2018)
- [MONAI: An open-source framework for deep learning in healthcare](https://mlsanity.com/paper/2211.02701/) (2022)
- [Segment Anything in High Quality](https://mlsanity.com/paper/2306.01567/) (2023)
- [Track Anything: Segment Anything Meets Videos](https://mlsanity.com/paper/2304.11968/) (2023)
- [Segment and Track Anything](https://mlsanity.com/paper/2305.06558/) (2023)
- [Tracking Everything Everywhere All at Once](https://mlsanity.com/paper/2306.05422/) (2023)
- [MedNeXt: Transformer-driven Scaling of ConvNets for Medical Image Segmentation](https://mlsanity.com/paper/2303.09975/) (2023)
- [MeshAnything: Artist-Created Mesh Generation with Autoregressive Transformers](https://mlsanity.com/paper/2406.10163/) (2024)
- [Deep Image Homography Estimation](https://mlsanity.com/paper/1606.03798/) (2016)
- [Weakly-Supervised Convolutional Neural Networks for Multimodal Image Registration](https://mlsanity.com/paper/1807.03361/) (2018)
- [DeepAtlas: Joint Semi-Supervised Learning of Image Registration and Segmentation](https://mlsanity.com/paper/1904.08465/) (2019)
- [Segment Anything in 3D with Radiance Fields](https://mlsanity.com/paper/2304.12308/) (2023)
- [uniGradICON: A Foundation Model for Medical Image Registration](https://mlsanity.com/paper/2403.05780/) (2024)
- [Dual-Stream Pyramid Registration Network](https://mlsanity.com/paper/1909.11966/) (2019)
- [textttGradICON: Approximate Diffeomorphisms via Gradient Inverse Consistency](https://mlsanity.com/paper/2206.05897/) (2022)
- [Affine Medical Image Registration with Coarse-to-Fine Vision Transformer](https://mlsanity.com/paper/2203.15216/) (2022)
- [Label-driven weakly-supervised learning for multimodal deformable image registration](https://mlsanity.com/paper/1711.01666/) (2017)
- [The Fully Convolutional Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2206.00566/) (2022)
- [SlimSAM: 0.1% Data Makes Segment Anything Slim](https://mlsanity.com/paper/2312.05284/) (2023)
- [Contrastive Registration for Unsupervised Medical Image Segmentation](https://mlsanity.com/paper/2011.08894/) (2020)
- [Joint Depth Prediction and Semantic Segmentation with Multi-View SAM](https://mlsanity.com/paper/2311.00134/) (2023)
- [Conditional Segmentation in Lieu of Image Registration](https://mlsanity.com/paper/1907.00438/) (2019)
- [One registration is worth two segmentations](https://mlsanity.com/paper/2405.10879/) (2024)
- [Diff-Reg v1: Diffusion Matching Model for Registration Problem](https://mlsanity.com/paper/2403.19919/) (2024)
- [Lightening Anything in Medical Images](https://mlsanity.com/paper/2406.10236/) (2024)
- [Segment Anything Model Meets Image Harmonization](https://mlsanity.com/paper/2312.12729/) (2023)

## Built on by (most cited first, 2 of 2)

- [Search-MIND: Training-Free Multi-Modal Medical Image Registration](https://mlsanity.com/paper/2604.09743/) (2026)
- [Markerless Tracking-Based Registration for Medical Image Motion Correction](https://mlsanity.com/paper/2503.10260/) (2025)

## Abstract (arXiv)

This paper describes a new spatial correspondence representation based on paired regions-of-interest (ROIs), for medical image registration. The distinct properties of the proposed ROI-based correspondence are discussed, in the context of potential benefits in clinical applications following image registration, compared with alternative correspondence-representing approaches, such as those based on sampled displacements and spatial transformation functions. These benefits include a clear connection between learning-based image registration and segmentation, which in turn motivates two cases of image registration approaches using (pre-)trained segmentation networks. Based on the segment anything model (SAM), a vision foundation model for segmentation, we develop a new registration algorithm SAMReg, which does not require any training (or training data), gradient-based fine-tuning or prompt engineering. The proposed SAMReg models are evaluated across five real-world applications, including intra-subject registration tasks with cardiac MR and lung CT, challenging inter-subject registration scenarios with prostate MR and retinal imaging, and an additional evaluation with a non-clinical example with aerial image registration. The proposed methods outperform both intensity-based iterative algorithms and DDF-predicting learning-based networks across tested metrics including Dice and target registration errors on anatomical structures, and further demonstrates competitive performance compared to weakly-supervised registration approaches that rely on fully-segmented training data. Open source code and examples are available at: https://github.com/sqhuang0103/SAMReg.git.