# Learning a Model-Driven Variational Network for Deformable Image Registration

arXiv: https://arxiv.org/abs/2105.12227 · HTML: https://mlsanity.com/paper/2105.12227/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 66 references that are in the map.
- First posted on arXiv 25 May 2021.

## Builds on (most cited first, 22 of 22)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [VoxelMorph: A Learning Framework for Deformable Medical Image Registration](https://mlsanity.com/paper/1809.05231/) (2018)
- [Learning Deep CNN Denoiser Prior for Image Restoration](https://mlsanity.com/paper/1704.03264/) (2017)
- [MoDL: Model Based Deep Learning Architecture for Inverse Problems](https://mlsanity.com/paper/1712.02862/) (2017)
- [Learning a Variational Network for Reconstruction of Accelerated MRI Data](https://mlsanity.com/paper/1704.00447/) (2017)
- [A Deep Cascade of Convolutional Neural Networks for Dynamic MR Image Reconstruction](https://mlsanity.com/paper/1704.02422/) (2017)
- [Trainable Nonlinear Reaction Diffusion: A Flexible Framework for Fast and Effective Image Restoration](https://mlsanity.com/paper/1508.02848/) (2015)
- [A Deep Learning Framework for Unsupervised Affine and Deformable Image Registration](https://mlsanity.com/paper/1809.06130/) (2018)
- [An Unsupervised Learning Model for Deformable Medical Image Registration](https://mlsanity.com/paper/1802.02604/) (2018)
- [Fast Symmetric Diffeomorphic Image Registration with Convolutional Neural Networks](https://mlsanity.com/paper/2003.09514/) (2020)
- [Recursive Cascaded Networks for Unsupervised Medical Image Registration](https://mlsanity.com/paper/1907.12353/) (2019)
- [Learning a Probabilistic Model for Diffeomorphic Registration](https://mlsanity.com/paper/1812.07460/) (2018)
- [Joint Learning of Motion Estimation and Segmentation for Cardiac MR Image Sequences](https://mlsanity.com/paper/1806.04066/) (2018)
- [End-to-End Learning of Motion Representation for Video Understanding](https://mlsanity.com/paper/1804.00413/) (2018)
- [Inverse-Consistent Deep Networks for Unsupervised Deformable Image Registration](https://mlsanity.com/paper/1809.03443/) (2018)
- [VS-Net: Variable splitting network for accelerated parallel MRI reconstruction](https://mlsanity.com/paper/1907.10033/) (2019)
- [mlVIRNET: Multilevel Variational Image Registration Network](https://mlsanity.com/paper/1909.10084/) (2019)
- [Model-Based and Data-Driven Strategies in Medical Image Computing](https://mlsanity.com/paper/1909.10391/) (2019)
- [A New Variational Model for Joint Image Reconstruction and Motion Estimation in Spatiotemporal Imaging](https://mlsanity.com/paper/1812.03446/) (2018)

## Built on by (most cited first, 25 of 25)

- [A survey on deep learning in medical image registration: new technologies, uncertainty, evaluation metrics, and beyond](https://mlsanity.com/paper/2307.15615/) (2023)
- [U-Net vs Transformer: Is U-Net Outdated in Medical Image Registration?](https://mlsanity.com/paper/2208.04939/) (2022)
- [Fourier-Net: Fast Image Registration with Band-limited Deformation](https://mlsanity.com/paper/2211.16342/) (2022)
- [Spatially Covariant Image Registration with Text Prompts](https://mlsanity.com/paper/2311.15607/) (2023)
- [Embedding Gradient-based Optimization in Image Registration Networks](https://mlsanity.com/paper/2112.03915/) (2021)
- [Recursive Deformable Image Registration Network with Mutual Attention](https://mlsanity.com/paper/2206.01863/) (2022)
- [GSMorph: Gradient Surgery for cine-MRI Cardiac Deformable Registration](https://mlsanity.com/paper/2306.14687/) (2023)
- [Decoder-Only Image Registration](https://mlsanity.com/paper/2402.03585/) (2024)
- [Fourier-Net+: Leveraging Band-Limited Representation for Efficient 3D Medical Image Registration](https://mlsanity.com/paper/2307.02997/) (2023)
- [WiNet: Wavelet-based Incremental Learning for Efficient Medical Image Registration](https://mlsanity.com/paper/2407.13426/) (2024)
- [Disentangling Progress in Medical Image Registration: Beyond Trend-Driven Architectures towards Domain-Specific Strategies](https://mlsanity.com/paper/2512.01913/) (2025)
- [Recovering high-quality FODs from a reduced number of diffusion-weighted images using a model-driven deep learning architecture](https://mlsanity.com/paper/2307.15273/) (2023)
- [Unsupervised Deformable Image Registration with Structural Nonparametric Smoothing](https://mlsanity.com/paper/2506.10813/) (2025)
- [Encoder-Only Image Registration](https://mlsanity.com/paper/2509.00451/) (2025)
- [FedMed-ATL: Misaligned Unpaired Brain Image Synthesis via Affine Transform Loss](https://mlsanity.com/paper/2201.12589/) (2022)
- [A lightweight residual network for unsupervised deformable image registration](https://mlsanity.com/paper/2406.09774/) (2024)
- [DARE: A Deformable Adaptive Regularization Estimator for Learning-Based Medical Image Registration](https://mlsanity.com/paper/2510.19353/) (2025)
- [Gaussian Primitive Optimized Deformable Retinal Image Registration](https://mlsanity.com/paper/2508.16852/) (2025)
- [Free-Boundary Quasiconformal Maps via a Least-squares Operator in Diffeomorphism Optimization](https://mlsanity.com/paper/2511.11679/) (2025)
- [Structure Unbiased Adversarial Model for Medical Image Segmentation](https://mlsanity.com/paper/2205.12857/) (2022)
- [Optimizing ADMM and Over-Relaxed ADMM Parameters for Linear Quadratic Problems](https://mlsanity.com/paper/2401.00657/) (2024)
- [An Efficient Model-Driven Groupwise Approach for Atlas Construction](https://mlsanity.com/paper/2508.10743/) (2025)
- [MorphSeek: Fine-grained Latent Representation-Level Policy Optimization for Deformable Image Registration](https://mlsanity.com/paper/2511.17392/) (2025)
- [Robust Rigid and Non-Rigid Medical Image Registration Using Learnable Edge Kernels](https://mlsanity.com/paper/2512.01771/) (2025)
- [Plasticine: A Traceable Diffusion Model for Medical Image Translation](https://mlsanity.com/paper/2512.18455/) (2025)

## Abstract (arXiv)

Data-driven deep learning approaches to image registration can be less accurate than conventional iterative approaches, especially when training data is limited. To address this whilst retaining the fast inference speed of deep learning, we propose VR-Net, a novel cascaded variational network for unsupervised deformable image registration. Using the variable splitting optimization scheme, we first convert the image registration problem, established in a generic variational framework, into two sub-problems, one with a point-wise, closed-form solution while the other one is a denoising problem. We then propose two neural layers (i.e. warping layer and intensity consistency layer) to model the analytical solution and a residual U-Net to formulate the denoising problem (i.e. generalized denoising layer). Finally, we cascade the warping layer, intensity consistency layer, and generalized denoising layer to form the VR-Net. Extensive experiments on three (two 2D and one 3D) cardiac magnetic resonance imaging datasets show that VR-Net outperforms state-of-the-art deep learning methods on registration accuracy, while maintains the fast inference speed of deep learning and the data-efficiency of variational model.