# Embedding Gradient-based Optimization in Image Registration Networks

arXiv: https://arxiv.org/abs/2112.03915 · HTML: https://mlsanity.com/paper/2112.03915/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 30 references that are in the map.
- First posted on arXiv 7 Dec 2021.

## Builds on (most cited first, 18 of 18)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [VoxelMorph: A Learning Framework for Deformable Medical Image Registration](https://mlsanity.com/paper/1809.05231/) (2018)
- [Learning a Variational Network for Reconstruction of Accelerated MRI Data](https://mlsanity.com/paper/1704.00447/) (2017)
- [Trainable Nonlinear Reaction Diffusion: A Flexible Framework for Fast and Effective Image Restoration](https://mlsanity.com/paper/1508.02848/) (2015)
- [A Deep Learning Framework for Unsupervised Affine and Deformable Image Registration](https://mlsanity.com/paper/1809.06130/) (2018)
- [Quicksilver: Fast Predictive Image Registration - a Deep Learning Approach](https://mlsanity.com/paper/1703.10908/) (2017)
- [Fast Symmetric Diffeomorphic Image Registration with Convolutional Neural Networks](https://mlsanity.com/paper/2003.09514/) (2020)
- [Recursive Cascaded Networks for Unsupervised Medical Image Registration](https://mlsanity.com/paper/1907.12353/) (2019)
- [Weakly-Supervised Convolutional Neural Networks for Multimodal Image Registration](https://mlsanity.com/paper/1807.03361/) (2018)
- [Image-and-Spatial Transformer Networks for Structure-Guided Image Registration](https://mlsanity.com/paper/1907.09200/) (2019)
- [Deep learning cardiac motion analysis for human survival prediction](https://mlsanity.com/paper/1810.03382/) (2018)
- [Metric Learning for Image Registration](https://mlsanity.com/paper/1904.09524/) (2019)
- [Learning a Model-Driven Variational Network for Deformable Image Registration](https://mlsanity.com/paper/2105.12227/) (2021)
- [mlVIRNET: Multilevel Variational Image Registration Network](https://mlsanity.com/paper/1909.10084/) (2019)
- [Learning Deformable Image Registration from Optimization: Perspective, Modules, Bilevel Training and Beyond](https://mlsanity.com/paper/2004.14557/) (2020)
- [Multi-scale Neural ODEs for 3D Medical Image Registration](https://mlsanity.com/paper/2106.08493/) (2021)
- [Recurrent Registration Neural Networks for Deformable Image Registration](https://mlsanity.com/paper/1906.09988/) (2019)

## Built on by (most cited first, 10 of 10)

- [Mamba? Catch The Hype Or Rethink What Really Helps for Image Registration](https://mlsanity.com/paper/2407.19274/) (2024)
- [DINOv3 with Test-Time Training for Medical Image Registration](https://mlsanity.com/paper/2508.14809/) (2025)
- [Conditional Deformable Image Registration with Spatially-Variant and Adaptive Regularization](https://mlsanity.com/paper/2303.10700/) (2023)
- [Deep Implicit Optimization enables Robust Learnable Features for Deformable Image Registration](https://mlsanity.com/paper/2406.07361/) (2024)
- [Fourier-Net+: Leveraging Band-Limited Representation for Efficient 3D Medical Image Registration](https://mlsanity.com/paper/2307.02997/) (2023)
- [Unsupervised Deformable Image Registration with Structural Nonparametric Smoothing](https://mlsanity.com/paper/2506.10813/) (2025)
- [Recurrent Inference Machine for Medical Image Registration](https://mlsanity.com/paper/2406.13413/) (2024)
- [CAR: Contrast-Agnostic Deformable Medical Image Registration with Contrast-Invariant Latent Regularization](https://mlsanity.com/paper/2408.05341/) (2024)
- [Bridging Classical and Learning-based Iterative Registration through Deep Equilibrium Models](https://mlsanity.com/paper/2507.00582/) (2025)
- [Gaussian Primitive Optimized Deformable Retinal Image Registration](https://mlsanity.com/paper/2508.16852/) (2025)

## Abstract (arXiv)

Deep learning (DL) image registration methods amortize the costly pair-wise iterative optimization by training deep neural networks to predict the optimal transformation in one fast forward-pass. In this work, we bridge the gap between traditional iterative energy optimization-based registration and network-based registration, and propose Gradient Descent Network for Image Registration (GraDIRN). Our proposed approach trains a DL network that embeds unrolled multiresolution gradient-based energy optimization in its forward pass, which explicitly enforces image dissimilarity minimization in its update steps. Extensive evaluations were performed on registration tasks using 2D cardiac MR and 3D brain MR images. We demonstrate that our approach achieved state-of-the-art registration performance while using fewer learned parameters, with good data efficiency and domain robustness.