# Affine Medical Image Registration with Coarse-to-Fine Vision Transformer

arXiv: https://arxiv.org/abs/2203.15216 · HTML: https://mlsanity.com/paper/2203.15216/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 48 references that are in the map.
- First posted on arXiv 29 Mar 2022.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions](https://mlsanity.com/paper/2102.12122/) (2021)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [CvT: Introducing Convolutions to Vision Transformers](https://mlsanity.com/paper/2103.15808/) (2021)
- [PVT v2: Improved Baselines with Pyramid Vision Transformer](https://mlsanity.com/paper/2106.13797/) (2021)
- [Network In Network](https://mlsanity.com/paper/1312.4400/) (2013)
- [CoAtNet: Marrying Convolution and Attention for All Data Sizes](https://mlsanity.com/paper/2106.04803/) (2021)
- [Twins: Revisiting the Design of Spatial Attention in Vision Transformers](https://mlsanity.com/paper/2104.13840/) (2021)
- [ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biases](https://mlsanity.com/paper/2103.10697/) (2021)
- [CMT: Convolutional Neural Networks Meet Vision Transformers](https://mlsanity.com/paper/2107.06263/) (2021)
- [An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solution](https://mlsanity.com/paper/1807.03247/) (2018)
- [LocalViT: Analyzing Locality in Vision Transformers](https://mlsanity.com/paper/2104.05707/) (2021)
- [A Deep Learning Framework for Unsupervised Affine and Deformable Image Registration](https://mlsanity.com/paper/1809.06130/) (2018)
- [An Unsupervised Learning Model for Deformable Medical Image Registration](https://mlsanity.com/paper/1802.02604/) (2018)
- [Visformer: The Vision-friendly Transformer](https://mlsanity.com/paper/2104.12533/) (2021)
- [Large Deformation Diffeomorphic Image Registration with Laplacian Pyramid Networks](https://mlsanity.com/paper/2006.16148/) (2020)
- [Unsupervised Learning for Fast Probabilistic Diffeomorphic Registration](https://mlsanity.com/paper/1805.04605/) (2018)
- [Fast Symmetric Diffeomorphic Image Registration with Convolutional Neural Networks](https://mlsanity.com/paper/2003.09514/) (2020)
- [ViT-V-Net: Vision Transformer for Unsupervised Volumetric Medical Image Registration](https://mlsanity.com/paper/2104.06468/) (2021)
- [HyperMorph: Amortized Hyperparameter Learning for Image Registration](https://mlsanity.com/paper/2101.01035/) (2021)
- [Unsupervised 3D End-to-End Medical Image Registration with Volume Tweening Network](https://mlsanity.com/paper/1902.05020/) (2019)
- [Conditional Deformable Image Registration with Convolutional Neural Network](https://mlsanity.com/paper/2106.12673/) (2021)
- [Networks for Joint Affine and Non-parametric Image Registration](https://mlsanity.com/paper/1903.08811/) (2019)
- [Closing the Gap between Deep and Conventional Image Registration using Probabilistic Dense Displacement Networks](https://mlsanity.com/paper/1907.10931/) (2019)
- [Label-driven weakly-supervised learning for multimodal deformable image registration](https://mlsanity.com/paper/1711.01666/) (2017)
- [CNN-based Lung CT Registration with Multiple Anatomical Constraints](https://mlsanity.com/paper/2011.14372/) (2020)
- [Weakly Supervised Registration of Prostate MRI and Histopathology Images](https://mlsanity.com/paper/2106.12526/) (2021)

## Built on by (most cited first, 30 of 35)

- [A survey on deep learning in medical image registration: new technologies, uncertainty, evaluation metrics, and beyond](https://mlsanity.com/paper/2307.15615/) (2023)
- [Transforming medical imaging with Transformers? A comparative review of key properties, current progresses, and future perspectives](https://mlsanity.com/paper/2206.01136/) (2022)
- [Deep Learning in Medical Image Registration: Magic or Mirage?](https://mlsanity.com/paper/2408.05839/) (2024)
- [Medical Image Registration via Neural Fields](https://mlsanity.com/paper/2206.03111/) (2022)
- [Anatomy-aware and acquisition-agnostic joint registration with SynthMorph](https://mlsanity.com/paper/2301.11329/) (2023)
- [Spatially Covariant Image Registration with Text Prompts](https://mlsanity.com/paper/2311.15607/) (2023)
- [MapLocNet: Coarse-to-Fine Feature Registration for Visual Re-Localization in Navigation Maps](https://mlsanity.com/paper/2407.08561/) (2024)
- [Transformers in Healthcare: A Survey](https://mlsanity.com/paper/2307.00067/) (2023)
- [Modality-Agnostic Structural Image Representation Learning for Deformable Multi-Modality Medical Image Registration](https://mlsanity.com/paper/2402.18933/) (2024)
- [Recent Progress in Transformer-based Medical Image Analysis](https://mlsanity.com/paper/2208.06643/) (2022)
- [Deformable Cross-Attention Transformer for Medical Image Registration](https://mlsanity.com/paper/2303.06179/) (2023)
- [Learning Accurate Template Matching with Differentiable Coarse-to-Fine Correspondence Refinement](https://mlsanity.com/paper/2303.08438/) (2023)
- [SAME++: A Self-supervised Anatomical eMbeddings Enhanced medical image registration framework using stable sampling and regularized transformation](https://mlsanity.com/paper/2311.14986/) (2023)
- [Registration by Regression (RbR): a framework for interpretable and flexible atlas registration](https://mlsanity.com/paper/2404.16781/) (2024)
- [MAD: Modality Agnostic Distance Measure for Image Registration](https://mlsanity.com/paper/2309.02875/) (2023)
- [Medical Image Registration and Its Application in Retinal Images: A Review](https://mlsanity.com/paper/2403.16502/) (2024)
- [Unsupervised Deformable Image Registration with Structural Nonparametric Smoothing](https://mlsanity.com/paper/2506.10813/) (2025)
- [InstantGroup: Instant Template Generation for Scalable Group of Brain MRI Registration](https://mlsanity.com/paper/2211.05622/) (2022)
- [Matching in the Wild: Learning Anatomical Embeddings for Multi-Modality Images](https://mlsanity.com/paper/2307.03535/) (2023)
- [SAMReg: SAM-enabled Image Registration with ROI-based Correspondence](https://mlsanity.com/paper/2410.14083/) (2024)
- [SACB-Net: Spatial-awareness Convolutions for Medical Image Registration](https://mlsanity.com/paper/2503.19592/) (2025)
- [SF2Former: Amyotrophic Lateral Sclerosis Identification From Multi-center MRI Data Using Spatial and Frequency Fusion Transformer](https://mlsanity.com/paper/2302.10859/) (2023)
- [On-the-Fly Guidance Training for Medical Image Registration](https://mlsanity.com/paper/2308.15216/) (2023)
- [Deep learning in medical image registration: introduction and survey](https://mlsanity.com/paper/2309.00727/) (2023)
- [QUIZ: An Arbitrary Volumetric Point Matching Method for Medical Image Registration](https://mlsanity.com/paper/2310.00296/) (2023)
- [UFM: Unified Feature Matching Pre-training with Multi-Modal Image Assistants](https://mlsanity.com/paper/2503.21820/) (2025)
- [Gaussian Primitive Optimized Deformable Retinal Image Registration](https://mlsanity.com/paper/2508.16852/) (2025)
- [A Scalable Distributed Framework for Multimodal GigaVoxel Image Registration](https://mlsanity.com/paper/2509.25044/) (2025)
- [A Two-Step Deep Learning Method for 3DCT-2DUS Kidney Registration During Breathing](https://mlsanity.com/paper/2305.13855/) (2023)
- [Adaptive Image Registration: A Hybrid Approach Integrating Deep Learning and Optimization Functions for Enhanced Precision](https://mlsanity.com/paper/2311.15497/) (2023)

## Abstract (arXiv)

Affine registration is indispensable in a comprehensive medical image registration pipeline. However, only a few studies focus on fast and robust affine registration algorithms. Most of these studies utilize convolutional neural networks (CNNs) to learn joint affine and non-parametric registration, while the standalone performance of the affine subnetwork is less explored. Moreover, existing CNN-based affine registration approaches focus either on the local misalignment or the global orientation and position of the input to predict the affine transformation matrix, which are sensitive to spatial initialization and exhibit limited generalizability apart from the training dataset. In this paper, we present a fast and robust learning-based algorithm, Coarse-to-Fine Vision Transformer (C2FViT), for 3D affine medical image registration. Our method naturally leverages the global connectivity and locality of the convolutional vision transformer and the multi-resolution strategy to learn the global affine registration. We evaluate our method on 3D brain atlas registration and template-matching normalization. Comprehensive results demonstrate that our method is superior to the existing CNNs-based affine registration methods in terms of registration accuracy, robustness and generalizability while preserving the runtime advantage of the learning-based methods. The source code is available at https://github.com/cwmok/C2FViT.