# ModeT: Learning Deformable Image Registration via Motion Decomposition Transformer

arXiv: https://arxiv.org/abs/2306.05688 · HTML: https://mlsanity.com/paper/2306.05688/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 25 references that are in the map.
- First posted on arXiv 9 Jun 2023.

## Builds on (most cited first, 12 of 12)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [VoxelMorph: A Learning Framework for Deformable Medical Image Registration](https://mlsanity.com/paper/1809.05231/) (2018)
- [Deep Learning in Medical Image Registration: A Review](https://mlsanity.com/paper/1912.12318/) (2019)
- [ViT-V-Net: Vision Transformer for Unsupervised Volumetric Medical Image Registration](https://mlsanity.com/paper/2104.06468/) (2021)
- [Dual-Stream Pyramid Registration Network](https://mlsanity.com/paper/1909.11966/) (2019)
- [XMorpher: Full Transformer for Deformable Medical Image Registration via Cross Attention](https://mlsanity.com/paper/2206.07349/) (2022)
- [Cross-modal Attention for MRI and Ultrasound Volume Registration](https://mlsanity.com/paper/2107.04548/) (2021)
- [Coordinate Translator for Learning Deformable Medical Image Registration](https://mlsanity.com/paper/2203.03626/) (2022)
- [Deformer: Towards Displacement Field Learning for Unsupervised Medical Image Registration](https://mlsanity.com/paper/2207.03180/) (2022)

## Built on by (most cited first, 23 of 23)

- [A survey on deep learning in medical image registration: new technologies, uncertainty, evaluation metrics, and beyond](https://mlsanity.com/paper/2307.15615/) (2023)
- [Vector Field Attention for Deformable Image Registration](https://mlsanity.com/paper/2407.10209/) (2024)
- [ModeTv2: GPU-accelerated Motion Decomposition Transformer for Pairwise Optimization in Medical Image Registration](https://mlsanity.com/paper/2403.16526/) (2024)
- [WiNet: Wavelet-based Incremental Learning for Efficient Medical Image Registration](https://mlsanity.com/paper/2407.13426/) (2024)
- [Disentangling Progress in Medical Image Registration: Beyond Trend-Driven Architectures towards Domain-Specific Strategies](https://mlsanity.com/paper/2512.01913/) (2025)
- [Medical Image Registration and Its Application in Retinal Images: A Review](https://mlsanity.com/paper/2403.16502/) (2024)
- [A lightweight residual network for unsupervised deformable image registration](https://mlsanity.com/paper/2406.09774/) (2024)
- [SACB-Net: Spatial-awareness Convolutions for Medical Image Registration](https://mlsanity.com/paper/2503.19592/) (2025)
- [DARE: A Deformable Adaptive Regularization Estimator for Learning-Based Medical Image Registration](https://mlsanity.com/paper/2510.19353/) (2025)
- [Bayesian Unsupervised Disentanglement of Anatomy and Geometry for Deep Groupwise Image Registration](https://mlsanity.com/paper/2401.02141/) (2024)
- [Pyramid Attention Network for Medical Image Registration](https://mlsanity.com/paper/2402.09016/) (2024)
- [Advancing Deformable Medical Image Registration with Multi-axis Cross-covariance Attention](https://mlsanity.com/paper/2412.18545/) (2024)
- [Implicit Deformable Medical Image Registration with Learnable Kernels](https://mlsanity.com/paper/2506.02150/) (2025)
- [Encoding Matching Criteria for Cross-domain Deformable Image Registration](https://mlsanity.com/paper/2406.12350/) (2024)
- [Dual-Attention Frequency Fusion at Multi-Scale for Joint Segmentation and Deformable Medical Image Registration](https://mlsanity.com/paper/2409.19658/) (2024)
- [FiRework: Field Refinement Framework for Efficient Enhancement of Deformable Registration](https://mlsanity.com/paper/2410.09595/) (2024)
- [Interpretable deformable image registration: A geometric deep learning perspective](https://mlsanity.com/paper/2412.13294/) (2024)
- [FF-PNet: A Pyramid Network Based on Feature and Field for Brain Image Registration](https://mlsanity.com/paper/2505.04938/) (2025)
- [Deformable Medical Image Registration with Effective Anatomical Structure Representation and Divide-and-Conquer Network](https://mlsanity.com/paper/2506.19222/) (2025)
- [MCM: Mamba-based Cardiac Motion Tracking using Sequential Images in MRI](https://mlsanity.com/paper/2507.17678/) (2025)
- [SHMoAReg: Spark Deformable Image Registration via Spatial Heterogeneous Mixture of Experts and Attention Heads](https://mlsanity.com/paper/2509.20073/) (2025)
- [Robust Rigid and Non-Rigid Medical Image Registration Using Learnable Edge Kernels](https://mlsanity.com/paper/2512.01771/) (2025)
- [Unsupervised Deformable Image Registration with Local-Global Attention and Image Decomposition](https://mlsanity.com/paper/2601.14337/) (2026)

## Abstract (arXiv)

The Transformer structures have been widely used in computer vision and have recently made an impact in the area of medical image registration. However, the use of Transformer in most registration networks is straightforward. These networks often merely use the attention mechanism to boost the feature learning as the segmentation networks do, but do not sufficiently design to be adapted for the registration task. In this paper, we propose a novel motion decomposition Transformer (ModeT) to explicitly model multiple motion modalities by fully exploiting the intrinsic capability of the Transformer structure for deformation estimation. The proposed ModeT naturally transforms the multi-head neighborhood attention relationship into the multi-coordinate relationship to model multiple motion modes. Then the competitive weighting module (CWM) fuses multiple deformation sub-fields to generate the resulting deformation field. Extensive experiments on two public brain magnetic resonance imaging (MRI) datasets show that our method outperforms current state-of-the-art registration networks and Transformers, demonstrating the potential of our ModeT for the challenging non-rigid deformation estimation problem. The benchmarks and our code are publicly available at https://github.com/ZAX130/SmileCode.