# Deep Learning in Medical Image Registration: Magic or Mirage?

arXiv: https://arxiv.org/abs/2408.05839 · HTML: https://mlsanity.com/paper/2408.05839/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 103 references that are in the map.
- First posted on arXiv 11 Aug 2024.

## Builds on (most cited first, 30 of 30)

- [Deep Image Prior](https://mlsanity.com/paper/1711.10925/) (2017)
- [A Deep Learning Framework for Unsupervised Affine and Deformable Image Registration](https://mlsanity.com/paper/1809.06130/) (2018)
- [SynthMorph: learning contrast-invariant registration without acquired images](https://mlsanity.com/paper/2004.10282/) (2020)
- [Large Deformation Diffeomorphic Image Registration with Laplacian Pyramid Networks](https://mlsanity.com/paper/2006.16148/) (2020)
- [Learn2Reg: comprehensive multi-task medical image registration challenge, dataset and evaluation in the era of deep learning](https://mlsanity.com/paper/2112.04489/) (2021)
- [Deep Learning in Medical Image Registration: A Survey](https://mlsanity.com/paper/1903.02026/) (2019)
- [Fast Symmetric Diffeomorphic Image Registration with Convolutional Neural Networks](https://mlsanity.com/paper/2003.09514/) (2020)
- [Recursive Cascaded Networks for Unsupervised Medical Image Registration](https://mlsanity.com/paper/1907.12353/) (2019)
- [Deep Learning in Medical Image Registration: A Review](https://mlsanity.com/paper/1912.12318/) (2019)
- [Coupling-based Invertible Neural Networks Are Universal Diffeomorphism Approximators](https://mlsanity.com/paper/2006.11469/) (2020)
- [HyperMorph: Amortized Hyperparameter Learning for Image Registration](https://mlsanity.com/paper/2101.01035/) (2021)
- [CycleMorph: Cycle Consistent Unsupervised Deformable Image Registration](https://mlsanity.com/paper/2008.05772/) (2020)
- [Deep Equilibrium Optical Flow Estimation](https://mlsanity.com/paper/2204.08442/) (2022)
- [Unsupervised 3D End-to-End Medical Image Registration with Volume Tweening Network](https://mlsanity.com/paper/1902.05020/) (2019)
- [Conditional Deformable Image Registration with Convolutional Neural Network](https://mlsanity.com/paper/2106.12673/) (2021)
- [Deep Distance Transform for Tubular Structure Segmentation in CT Scans](https://mlsanity.com/paper/1912.03383/) (2019)
- [textttGradICON: Approximate Diffeomorphisms via Gradient Inverse Consistency](https://mlsanity.com/paper/2206.05897/) (2022)
- [Fast 3D registration with accurate optimisation and little learning for Learn2Reg 2021](https://mlsanity.com/paper/2112.03053/) (2021)
- [U-Net vs Transformer: Is U-Net Outdated in Medical Image Registration?](https://mlsanity.com/paper/2208.04939/) (2022)
- [Affine Medical Image Registration with Coarse-to-Fine Vision Transformer](https://mlsanity.com/paper/2203.15216/) (2022)
- [NODEO: A Neural Ordinary Differential Equation Based Optimization Framework for Deformable Image Registration](https://mlsanity.com/paper/2108.03443/) (2021)
- [Biomechanics-informed Neural Networks for Myocardial Motion Tracking in MRI](https://mlsanity.com/paper/2006.04725/) (2020)
- [Unsupervised Deformable Image Registration Using Cycle-Consistent CNN](https://mlsanity.com/paper/1907.01319/) (2019)
- [Generative Myocardial Motion Tracking via Latent Space Exploration with Biomechanics-informed Prior](https://mlsanity.com/paper/2206.03830/) (2022)
- [SAME: Deformable Image Registration based on Self-supervised Anatomical Embeddings](https://mlsanity.com/paper/2109.11572/) (2021)
- [A Lagrangian Gauss-Newton-Krylov Solver for Mass- and Intensity-Preserving Diffeomorphic Image Registration](https://mlsanity.com/paper/1703.04446/) (2017)
- [SAME++: A Self-supervised Anatomical eMbeddings Enhanced medical image registration framework using stable sampling and regularized transformation](https://mlsanity.com/paper/2311.14986/) (2023)
- [CLAIRE: A distributed-memory solver for constrained large deformation diffeomorphic image registration](https://mlsanity.com/paper/1808.04487/) (2018)
- [ssEMnet: Serial-section Electron Microscopy Image Registration using a Spatial Transformer Network with Learned Features](https://mlsanity.com/paper/1707.07833/) (2017)
- [Learning deep abdominal CT registration through adaptive loss weighting and synthetic data generation](https://mlsanity.com/paper/2211.15717/) (2022)

## Built on by (most cited first, 21 of 21)

- [Beyond the LUMIR challenge: The pathway to foundational registration models](https://mlsanity.com/paper/2505.24160/) (2025)
- [IMPACT: A Generic Semantic Loss for Multimodal Medical Image Registration](https://mlsanity.com/paper/2503.24121/) (2025)
- [Disentangling Progress in Medical Image Registration: Beyond Trend-Driven Architectures towards Domain-Specific Strategies](https://mlsanity.com/paper/2512.01913/) (2025)
- [Unsupervised Deformable Image Registration with Structural Nonparametric Smoothing](https://mlsanity.com/paper/2506.10813/) (2025)
- [VoxelOpt: Voxel-Adaptive Message Passing for Discrete Optimization in Deformable Abdominal CT Registration](https://mlsanity.com/paper/2506.19975/) (2025)
- [The LUMirage: An independent evaluation of zero-shot performance in the LUMIR challenge](https://mlsanity.com/paper/2512.15505/) (2025)
- [Evaluation of Deformable Image Registration under Alignment-Regularity Trade-off](https://mlsanity.com/paper/2503.07185/) (2025)
- [A Scalable Distributed Framework for Multimodal GigaVoxel Image Registration](https://mlsanity.com/paper/2509.25044/) (2025)
- [Strategies for Robust Deep Learning Based Deformable Registration](https://mlsanity.com/paper/2510.23079/) (2025)
- [MUSTER: Longitudinal Deformable Registration by Composition of Consecutive Deformations](https://mlsanity.com/paper/2412.14671/) (2024)
- [Rapid patient-specific neural networks for intraoperative X-ray to volume registration](https://mlsanity.com/paper/2503.16309/) (2025)
- [Biomechanical Constraints Assimilation in Deep-Learning Image Registration: Application to sliding and locally rigid deformations](https://mlsanity.com/paper/2504.05444/) (2025)
- [BodyGPS: Anatomical Positioning System](https://mlsanity.com/paper/2505.07744/) (2025)
- [Consistent Point Matching](https://mlsanity.com/paper/2507.23609/) (2025)
- [Ideal Registration? Segmentation is All You Need](https://mlsanity.com/paper/2509.15784/) (2025)
- [Deformable Medical Image Registration with KAN-based Implicit Neural Representations](https://mlsanity.com/paper/2509.22874/) (2025)
- [MGRegBench: A Novel Benchmark Dataset with Anatomical Landmarks for Mammography Image Registration](https://mlsanity.com/paper/2512.17605/) (2025)
- [The Right Prior for the Right Deformation: Rethinking Continuous Deformable Image Registration](https://mlsanity.com/paper/2608.16146/) (2026)
- [Is Deformable Image Registration Ready for Brain Metastasis Reirradiation Dose Accumulation? A Longitudinal MRI Benchmark of Registration Accuracy](https://mlsanity.com/paper/2608.28705/) (2026)
- [BINDER: A Latent Variable Model for Probabilistic Medical Image Registration](https://mlsanity.com/paper/2609.19875/) (2026)
- [Registration is a Representation Problem: Solving Whole-Body PET/CT Registration](https://mlsanity.com/paper/2610.18804/) (2026)

## Abstract (arXiv)

Classical optimization and learning-based methods are the two reigning paradigms in deformable image registration. While optimization-based methods boast generalizability across modalities and robust performance, learning-based methods promise peak performance, incorporating weak supervision and amortized optimization. However, the exact conditions for either paradigm to perform well over the other are shrouded and not explicitly outlined in the existing literature. In this paper, we make an explicit correspondence between the mutual information of the distribution of per-pixel intensity and labels, and the performance of classical registration methods. This strong correlation hints to the fact that architectural designs in learning-based methods is unlikely to affect this correlation, and therefore, the performance of learning-based methods. This hypothesis is thoroughly validated with state-of-the-art classical and learning-based methods. However, learning-based methods with weak supervision can perform high-fidelity intensity and label registration, which is not possible with classical methods. Next, we show that this high-fidelity feature learning does not translate to invariance to domain shift, and learning-based methods are sensitive to such changes in the data distribution. Finally, we propose a general recipe to choose the best paradigm for a given registration problem, based on these observations.