# Decoder-Only Image Registration

arXiv: https://arxiv.org/abs/2402.03585 · HTML: https://mlsanity.com/paper/2402.03585/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 68 references that are in the map.
- First posted on arXiv 5 Feb 2024.

## Builds on (most cited first, 30 of 30)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [VoxelMorph: A Learning Framework for Deformable Medical Image Registration](https://mlsanity.com/paper/1809.05231/) (2018)
- [CHAOS Challenge -- Combined (CT-MR) Healthy Abdominal Organ Segmentation](https://mlsanity.com/paper/2001.06535/) (2020)
- [Unsupervised Learning of Probabilistic Diffeomorphic Registration for Images and Surfaces](https://mlsanity.com/paper/1903.03545/) (2019)
- [A Deep Learning Framework for Unsupervised Affine and Deformable Image Registration](https://mlsanity.com/paper/1809.06130/) (2018)
- [An Unsupervised Learning Model for Deformable Medical Image Registration](https://mlsanity.com/paper/1802.02604/) (2018)
- [Large Deformation Diffeomorphic Image Registration with Laplacian Pyramid Networks](https://mlsanity.com/paper/2006.16148/) (2020)
- [Unsupervised Learning for Fast Probabilistic Diffeomorphic Registration](https://mlsanity.com/paper/1805.04605/) (2018)
- [Fast Symmetric Diffeomorphic Image Registration with Convolutional Neural Networks](https://mlsanity.com/paper/2003.09514/) (2020)
- [Recursive Cascaded Networks for Unsupervised Medical Image Registration](https://mlsanity.com/paper/1907.12353/) (2019)
- [ViT-V-Net: Vision Transformer for Unsupervised Volumetric Medical Image Registration](https://mlsanity.com/paper/2104.06468/) (2021)
- [HyperMorph: Amortized Hyperparameter Learning for Image Registration](https://mlsanity.com/paper/2101.01035/) (2021)
- [CycleMorph: Cycle Consistent Unsupervised Deformable Image Registration](https://mlsanity.com/paper/2008.05772/) (2020)
- [End-to-End Unsupervised Deformable Image Registration with a Convolutional Neural Network](https://mlsanity.com/paper/1704.06065/) (2017)
- [Learning a Probabilistic Model for Diffeomorphic Registration](https://mlsanity.com/paper/1812.07460/) (2018)
- [Dual-Stream Pyramid Registration Network](https://mlsanity.com/paper/1909.11966/) (2019)
- [Joint Learning of Motion Estimation and Segmentation for Cardiac MR Image Sequences](https://mlsanity.com/paper/1806.04066/) (2018)
- [U-Net vs Transformer: Is U-Net Outdated in Medical Image Registration?](https://mlsanity.com/paper/2208.04939/) (2022)
- [Learning the Effect of Registration Hyperparameters with HyperMorph](https://mlsanity.com/paper/2203.16680/) (2022)
- [Inverse-Consistent Deep Networks for Unsupervised Deformable Image Registration](https://mlsanity.com/paper/1809.03443/) (2018)
- [Learning a Model-Driven Variational Network for Deformable Image Registration](https://mlsanity.com/paper/2105.12227/) (2021)
- [Non-Rigid Image Registration Using Self-Supervised Fully Convolutional Networks without Training Data](https://mlsanity.com/paper/1801.04012/) (2018)
- [Fourier-Net: Fast Image Registration with Band-limited Deformation](https://mlsanity.com/paper/2211.16342/) (2022)
- [FAIM -- A ConvNet Method for Unsupervised 3D Medical Image Registration](https://mlsanity.com/paper/1811.09243/) (2018)
- [Learning Deformable Image Registration from Optimization: Perspective, Modules, Bilevel Training and Beyond](https://mlsanity.com/paper/2004.14557/) (2020)
- [Contrastive Registration for Unsupervised Medical Image Segmentation](https://mlsanity.com/paper/2011.08894/) (2020)
- [Model-Based and Data-Driven Strategies in Medical Image Computing](https://mlsanity.com/paper/1909.10391/) (2019)
- [Nesterov Accelerated ADMM for Fast Diffeomorphic Image Registration](https://mlsanity.com/paper/2109.12688/) (2021)
- [Fourier-Net+: Leveraging Band-Limited Representation for Efficient 3D Medical Image Registration](https://mlsanity.com/paper/2307.02997/) (2023)

## Built on by (most cited first, 5 of 5)

- [Encoder-Only Image Registration](https://mlsanity.com/paper/2509.00451/) (2025)
- [Efficient Large-Deformation Medical Image Registration via Recurrent Dynamic Correlation](https://mlsanity.com/paper/2510.22380/) (2025)
- [Plasticine: A Traceable Diffusion Model for Medical Image Translation](https://mlsanity.com/paper/2512.18455/) (2025)
- [Neural Phase Correlation](https://mlsanity.com/paper/2606.18496/) (2026)
- [DrivenMorph: Bridging Attention Mechanism and Variational Image Registration via Difference Modeling](https://mlsanity.com/paper/2606.30183/) (2026)

## Abstract (arXiv)

In unsupervised medical image registration, the predominant approaches involve the utilization of a encoder-decoder network architecture, allowing for precise prediction of dense, full-resolution displacement fields from given paired images. Despite its widespread use in the literature, we argue for the necessity of making both the encoder and decoder learnable in such an architecture. For this, we propose a novel network architecture, termed LessNet in this paper, which contains only a learnable decoder, while entirely omitting the utilization of a learnable encoder. LessNet substitutes the learnable encoder with simple, handcrafted features, eliminating the need to learn (optimize) network parameters in the encoder altogether. Consequently, this leads to a compact, efficient, and decoder-only architecture for 3D medical image registration. Evaluated on two publicly available brain MRI datasets, we demonstrate that our decoder-only LessNet can effectively and efficiently learn both dense displacement and diffeomorphic deformation fields in 3D. Furthermore, our decoder-only LessNet can achieve comparable registration performance to state-of-the-art methods such as VoxelMorph and TransMorph, while requiring significantly fewer computational resources. Our code and pre-trained models are available at https://github.com/xi-jia/LessNet.