# PC-SwinMorph: Patch Representation for Unsupervised Medical Image Registration and Segmentation

arXiv: https://arxiv.org/abs/2203.05684 · HTML: https://mlsanity.com/paper/2203.05684/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 44 references that are in the map.
- First posted on arXiv 10 Mar 2022.

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions](https://mlsanity.com/paper/2102.12122/) (2021)
- [TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation](https://mlsanity.com/paper/2102.04306/) (2021)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [Contrastive Multiview Coding](https://mlsanity.com/paper/1906.05849/) (2019)
- [UNETR: Transformers for 3D Medical Image Segmentation](https://mlsanity.com/paper/2103.10504/) (2021)
- [VoxelMorph: A Learning Framework for Deformable Medical Image Registration](https://mlsanity.com/paper/1809.05231/) (2018)
- [Transformer in Transformer](https://mlsanity.com/paper/2103.00112/) (2021)
- [A Deep Learning Framework for Unsupervised Affine and Deformable Image Registration](https://mlsanity.com/paper/1809.06130/) (2018)
- [An Unsupervised Learning Model for Deformable Medical Image Registration](https://mlsanity.com/paper/1802.02604/) (2018)
- [Quicksilver: Fast Predictive Image Registration - a Deep Learning Approach](https://mlsanity.com/paper/1703.10908/) (2017)
- [Fast Symmetric Diffeomorphic Image Registration with Convolutional Neural Networks](https://mlsanity.com/paper/2003.09514/) (2020)
- [Recursive Cascaded Networks for Unsupervised Medical Image Registration](https://mlsanity.com/paper/1907.12353/) (2019)
- [Contrastive Learning for Label-Efficient Semantic Segmentation](https://mlsanity.com/paper/2012.06985/) (2020)
- [ViT-V-Net: Vision Transformer for Unsupervised Volumetric Medical Image Registration](https://mlsanity.com/paper/2104.06468/) (2021)
- [CycleMorph: Cycle Consistent Unsupervised Deformable Image Registration](https://mlsanity.com/paper/2008.05772/) (2020)
- [End-to-End Unsupervised Deformable Image Registration with a Convolutional Neural Network](https://mlsanity.com/paper/1704.06065/) (2017)
- [Networks for Joint Affine and Non-parametric Image Registration](https://mlsanity.com/paper/1903.08811/) (2019)
- [LT-Net: Label Transfer by Learning Reversible Voxel-wise Correspondence for One-shot Medical Image Segmentation](https://mlsanity.com/paper/2003.07072/) (2020)
- [DeepTag: An Unsupervised Deep Learning Method for Motion Tracking on Cardiac Tagging Magnetic Resonance Images](https://mlsanity.com/paper/2103.02772/) (2021)
- [Probabilistic Multilayer Regularization Network for Unsupervised 3D Brain Image Registration](https://mlsanity.com/paper/1907.01922/) (2019)
- [Rethinking Medical Image Reconstruction via Shape Prior, Going Deeper and Faster: Deep Joint Indirect Registration and Reconstruction](https://mlsanity.com/paper/1912.07648/) (2019)

## Built on by (most cited first, 8 of 8)

- [A survey on deep learning in medical image registration: new technologies, uncertainty, evaluation metrics, and beyond](https://mlsanity.com/paper/2307.15615/) (2023)
- [Transforming medical imaging with Transformers? A comparative review of key properties, current progresses, and future perspectives](https://mlsanity.com/paper/2206.01136/) (2022)
- [SCOTCH and SODA: A Transformer Video Shadow Detection Framework](https://mlsanity.com/paper/2211.06885/) (2022)
- [UniBrain: Universal Brain MRI Diagnosis with Hierarchical Knowledge-enhanced Pre-training](https://mlsanity.com/paper/2309.06828/) (2023)
- [Biophysics Informed Pathological Regularisation for Brain Tumour Segmentation](https://mlsanity.com/paper/2403.09136/) (2024)
- [Learning Homeomorphic Image Registration via Conformal-Invariant Hyperelastic Regularisation](https://mlsanity.com/paper/2303.08113/) (2023)
- [DmADs-Net: Dense multiscale attention and depth-supervised network for medical image segmentation](https://mlsanity.com/paper/2405.00472/) (2024)
- [Optimised ProPainter for Video Diminished Reality Inpainting](https://mlsanity.com/paper/2406.02287/) (2024)

## Abstract (arXiv)

Medical image registration and segmentation are critical tasks for several clinical procedures. Manual realisation of those tasks is time-consuming and the quality is highly dependent on the level of expertise of the physician. To mitigate that laborious task, automatic tools have been developed where the majority of solutions are supervised techniques. However, in medical domain, the strong assumption of having a well-representative ground truth is far from being realistic. To overcome this challenge, unsupervised techniques have been investigated. However, they are still limited in performance and they fail to produce plausible results. In this work, we propose a novel unified unsupervised framework for image registration and segmentation that we called PC-SwinMorph. The core of our framework is two patch-based strategies, where we demonstrate that patch representation is key for performance gain. We first introduce a patch-based contrastive strategy that enforces locality conditions and richer feature representation. Secondly, we utilise a 3D window/shifted-window multi-head self-attention module as a patch stitching strategy to eliminate artifacts from the patch splitting. We demonstrate, through a set of numerical and visual results, that our technique outperforms current state-of-the-art unsupervised techniques.