# 3D Inception-Based TransMorph: Pre- and Post-operative Multi-contrast MRI Registration in Brain Tumors

arXiv: https://arxiv.org/abs/2212.04579 · HTML: https://mlsanity.com/paper/2212.04579/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 34 references that are in the map.
- First posted on arXiv 8 Dec 2022.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [VoxelMorph: A Learning Framework for Deformable Medical Image Registration](https://mlsanity.com/paper/1809.05231/) (2018)
- [Hierarchical Multi-Scale Attention for Semantic Segmentation](https://mlsanity.com/paper/2005.10821/) (2020)
- [Attention to Scale: Scale-aware Semantic Image Segmentation](https://mlsanity.com/paper/1511.03339/) (2015)
- [An Unsupervised Learning Model for Deformable Medical Image Registration](https://mlsanity.com/paper/1802.02604/) (2018)
- [Unsupervised Learning for Fast Probabilistic Diffeomorphic Registration](https://mlsanity.com/paper/1805.04605/) (2018)
- [Quicksilver: Fast Predictive Image Registration - a Deep Learning Approach](https://mlsanity.com/paper/1703.10908/) (2017)
- [Rethinking Classification and Localization for Object Detection](https://mlsanity.com/paper/1904.06493/) (2019)
- [ViT-V-Net: Vision Transformer for Unsupervised Volumetric Medical Image Registration](https://mlsanity.com/paper/2104.06468/) (2021)
- [Scale-aware Fast R-CNN for Pedestrian Detection](https://mlsanity.com/paper/1510.08160/) (2015)
- [The Brain Tumor Sequence Registration (BraTS-Reg) Challenge: Establishing Correspondence Between Pre-Operative and Follow-up MRI Scans of Diffuse Glioma Patients](https://mlsanity.com/paper/2112.06979/) (2021)
- [CFUN: Combining Faster R-CNN and U-net Network for Efficient Whole Heart Segmentation](https://mlsanity.com/paper/1812.04914/) (2018)

## Built on by (most cited first, 5 of 5)

- [An Ensemble Approach for Brain Tumor Segmentation and Synthesis](https://mlsanity.com/paper/2411.17617/) (2024)
- [An Optimization Framework for Processing and Transfer Learning for the Brain Tumor Segmentation](https://mlsanity.com/paper/2402.07008/) (2024)
- [Primitive Simultaneous Optimization of Similarity Metrics for Image Registration](https://mlsanity.com/paper/2304.01601/) (2023)
- [SFL-Net: Source-Factorized Latent Representation Learning for Multi-Contrast MRI to Tau-PET Synthesis](https://mlsanity.com/paper/2602.22545/) (2026)
- [Data-driven registration and modeling of brain deformation for image-guided neurosurgery](https://mlsanity.com/paper/2602.10155/) (2026)

## Abstract (arXiv)

Deformable image registration is a key task in medical image analysis. The Brain Tumor Sequence Registration challenge (BraTS-Reg) aims at establishing correspondences between pre-operative and follow-up scans of the same patient diagnosed with an adult brain diffuse high-grade glioma and intends to address the challenging task of registering longitudinal data with major tissue appearance changes. In this work, we proposed a two-stage cascaded network based on the Inception and TransMorph models. The dataset for each patient was comprised of a native pre-contrast (T1), a contrast-enhanced T1-weighted (T1-CE), a T2-weighted (T2), and a Fluid Attenuated Inversion Recovery (FLAIR). The Inception model was used to fuse the 4 image modalities together and extract the most relevant information. Then, a variant of the TransMorph architecture was adapted to generate the displacement fields. The Loss function was composed of a standard image similarity measure, a diffusion regularizer, and an edge-map similarity measure added to overcome intensity dependence and reinforce correct boundary deformation. We observed that the addition of the Inception module substantially increased the performance of the network. Additionally, performing an initial affine registration before training the model showed improved accuracy in the landmark error measurements between pre and post-operative MRIs. We observed that our best model composed of the Inception and TransMorph architectures while using an initially affine registered dataset had the best performance with a median absolute error of 2.91 (initial error = 7.8). We achieved 6th place at the time of model submission in the final testing phase of the BraTS-Reg challenge.