# ModeTv2: GPU-accelerated Motion Decomposition Transformer for Pairwise Optimization in Medical Image Registration

arXiv: https://arxiv.org/abs/2403.16526 · HTML: https://mlsanity.com/paper/2403.16526/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 81 references that are in the map.
- First posted on arXiv 25 Mar 2024.

## Builds on (most cited first, 30 of 37)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [VoxelMorph: A Learning Framework for Deformable Medical Image Registration](https://mlsanity.com/paper/1809.05231/) (2018)
- [Unsupervised Learning of Probabilistic Diffeomorphic Registration for Images and Surfaces](https://mlsanity.com/paper/1903.03545/) (2019)
- [Large Deformation Diffeomorphic Image Registration with Laplacian Pyramid Networks](https://mlsanity.com/paper/2006.16148/) (2020)
- [Deep Learning in Medical Image Registration: A Survey](https://mlsanity.com/paper/1903.02026/) (2019)
- [Recursive Cascaded Networks for Unsupervised Medical Image Registration](https://mlsanity.com/paper/1907.12353/) (2019)
- [Deep Learning in Medical Image Registration: A Review](https://mlsanity.com/paper/1912.12318/) (2019)
- [ViT-V-Net: Vision Transformer for Unsupervised Volumetric Medical Image Registration](https://mlsanity.com/paper/2104.06468/) (2021)
- [Unsupervised 3D End-to-End Medical Image Registration with Volume Tweening Network](https://mlsanity.com/paper/1902.05020/) (2019)
- [Dual-Stream Pyramid Registration Network](https://mlsanity.com/paper/1909.11966/) (2019)
- [XMorpher: Full Transformer for Deformable Medical Image Registration via Cross Attention](https://mlsanity.com/paper/2206.07349/) (2022)
- [mlVIRNET: Multilevel Variational Image Registration Network](https://mlsanity.com/paper/1909.10084/) (2019)
- [ModeT: Learning Deformable Image Registration via Motion Decomposition Transformer](https://mlsanity.com/paper/2306.05688/) (2023)
- [Joint Progressive and Coarse-to-fine Registration of Brain MRI via Deformation Field Integration and Non-Rigid Feature Fusion](https://mlsanity.com/paper/2109.12384/) (2021)
- [Slice-to-volume medical image registration: a survey](https://mlsanity.com/paper/1702.01636/) (2017)
- [One Shot Learning for Deformable Medical Image Registration and Periodic Motion Tracking](https://mlsanity.com/paper/1907.04641/) (2019)
- [Cross-modal Attention for MRI and Ultrasound Volume Registration](https://mlsanity.com/paper/2107.04548/) (2021)
- [Non-iterative Coarse-to-fine Transformer Networks for Joint Affine and Deformable Image Registration](https://mlsanity.com/paper/2307.03421/) (2023)
- [Coordinate Translator for Learning Deformable Medical Image Registration](https://mlsanity.com/paper/2203.03626/) (2022)
- [Non-iterative Coarse-to-fine Registration based on Single-pass Deep Cumulative Learning](https://mlsanity.com/paper/2206.12596/) (2022)

## Built on by (most cited first, 4 of 4)

- [Beyond the LUMIR challenge: The pathway to foundational registration models](https://mlsanity.com/paper/2505.24160/) (2025)
- [Encoding Matching Criteria for Cross-domain Deformable Image Registration](https://mlsanity.com/paper/2406.12350/) (2024)
- [FiRework: Field Refinement Framework for Efficient Enhancement of Deformable Registration](https://mlsanity.com/paper/2410.09595/) (2024)
- [Accelerated 3D-3D rigid registration of echocardiographic images obtained from apical window using particle filter](https://mlsanity.com/paper/2504.19930/) (2025)

## Abstract (arXiv)

Deformable image registration plays a crucial role in medical imaging, aiding in disease diagnosis and image-guided interventions. Traditional iterative methods are slow, while deep learning (DL) accelerates solutions but faces usability and precision challenges. This study introduces a pyramid network with the enhanced motion decomposition Transformer (ModeTv2) operator, showcasing superior pairwise optimization (PO) akin to traditional methods. We re-implement ModeT operator with CUDA extensions to enhance its computational efficiency. We further propose RegHead module which refines deformation fields, improves the realism of deformation and reduces parameters. By adopting the PO, the proposed network balances accuracy, efficiency, and generalizability. Extensive experiments on three public brain MRI datasets and one abdominal CT dataset demonstrate the network's suitability for PO, providing a DL model with enhanced usability and interpretability. The code is publicly available at https://github.com/ZAX130/ModeTv2.