# One registration is worth two segmentations

arXiv: https://arxiv.org/abs/2405.10879 · HTML: https://mlsanity.com/paper/2405.10879/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 31 references that are in the map.
- First posted on arXiv 17 May 2024.

## Builds on (most cited first, 16 of 16)

- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [VoxelMorph: A Learning Framework for Deformable Medical Image Registration](https://mlsanity.com/paper/1809.05231/) (2018)
- [Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentation](https://mlsanity.com/paper/2304.12620/) (2023)
- [MONAI: An open-source framework for deep learning in healthcare](https://mlsanity.com/paper/2211.02701/) (2022)
- [AMOS: A Large-Scale Abdominal Multi-Organ Benchmark for Versatile Medical Image Segmentation](https://mlsanity.com/paper/2206.08023/) (2022)
- [Segment Anything Model for Medical Image Analysis: an Experimental Study](https://mlsanity.com/paper/2304.10517/) (2023)
- [MedNeXt: Transformer-driven Scaling of ConvNets for Medical Image Segmentation](https://mlsanity.com/paper/2303.09975/) (2023)
- [A Deep Learning Framework for Unsupervised Affine and Deformable Image Registration](https://mlsanity.com/paper/1809.06130/) (2018)
- [Deep Image Homography Estimation](https://mlsanity.com/paper/1606.03798/) (2016)
- [SAM-Med3D: Towards General-purpose Segmentation Models for Volumetric Medical Images](https://mlsanity.com/paper/2310.15161/) (2023)
- [Weakly-Supervised Convolutional Neural Networks for Multimodal Image Registration](https://mlsanity.com/paper/1807.03361/) (2018)
- [3DSAM-adapter: Holistic adaptation of SAM from 2D to 3D for promptable tumor segmentation](https://mlsanity.com/paper/2306.13465/) (2023)
- [The Fully Convolutional Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2206.00566/) (2022)
- [Neural Multi-Atlas Label Fusion: Application to Cardiac MR Images](https://mlsanity.com/paper/1709.09641/) (2017)
- [Conditional Segmentation in Lieu of Image Registration](https://mlsanity.com/paper/1907.00438/) (2019)

## Built on by (most cited first, 6 of 6)

- [SAMReg: SAM-enabled Image Registration with ROI-based Correspondence](https://mlsanity.com/paper/2410.14083/) (2024)
- [Tell2Reg: Establishing spatial correspondence between images by the same language prompts](https://mlsanity.com/paper/2502.03118/) (2025)
- [Deformable Medical Image Registration with Effective Anatomical Structure Representation and Divide-and-Conquer Network](https://mlsanity.com/paper/2506.19222/) (2025)
- [Register Anything: Estimating "Corresponding Prompts" for Segment Anything Model](https://mlsanity.com/paper/2508.01697/) (2025)
- [Reg-TTR, Test-Time Refinement for Fast, Robust and Accurate Image Registration](https://mlsanity.com/paper/2601.19114/) (2026)
- [GLIDE-Reg: Global-to-Local Deformable Registration Using Co-Optimized Foundation and Handcrafted Features](https://mlsanity.com/paper/2603.00218/) (2026)

## Abstract (arXiv)

The goal of image registration is to establish spatial correspondence between two or more images, traditionally through dense displacement fields (DDFs) or parametric transformations (e.g., rigid, affine, and splines). Rethinking the existing paradigms of achieving alignment via spatial transformations, we uncover an alternative but more intuitive correspondence representation: a set of corresponding regions-of-interest (ROI) pairs, which we demonstrate to have sufficient representational capability as other correspondence representation methods.Further, it is neither necessary nor sufficient for these ROIs to hold specific anatomical or semantic significance. In turn, we formulate image registration as searching for the same set of corresponding ROIs from both moving and fixed images - in other words, two multi-class segmentation tasks on a pair of images. For a general-purpose and practical implementation, we integrate the segment anything model (SAM) into our proposed algorithms, resulting in a SAM-enabled registration (SAMReg) that does not require any training data, gradient-based fine-tuning or engineered prompts. We experimentally show that the proposed SAMReg is capable of segmenting and matching multiple ROI pairs, which establish sufficiently accurate correspondences, in three clinical applications of registering prostate MR, cardiac MR and abdominal CT images. Based on metrics including Dice and target registration errors on anatomical structures, the proposed registration outperforms both intensity-based iterative algorithms and DDF-predicting learning-based networks, even yielding competitive performance with weakly-supervised registration which requires fully-segmented training data.