# AutoFuse: Automatic Fusion Networks for Deformable Medical Image Registration

arXiv: https://arxiv.org/abs/2309.05271 · HTML: https://mlsanity.com/paper/2309.05271/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 66 references that are in the map.
- First posted on arXiv 11 Sep 2023.

## Builds on (most cited first, 29 of 29)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2105.05537/) (2021)
- [VoxelMorph: A Learning Framework for Deformable Medical Image Registration](https://mlsanity.com/paper/1809.05231/) (2018)
- [Cross-stitch Networks for Multi-task Learning](https://mlsanity.com/paper/1604.03539/) (2016)
- [Transformers in Medical Imaging: A Survey](https://mlsanity.com/paper/2201.09873/) (2022)
- [Unsupervised Learning of Probabilistic Diffeomorphic Registration for Images and Surfaces](https://mlsanity.com/paper/1903.03545/) (2019)
- [A Deep Learning Framework for Unsupervised Affine and Deformable Image Registration](https://mlsanity.com/paper/1809.06130/) (2018)
- [Deep Learning in Medical Image Registration: A Survey](https://mlsanity.com/paper/1903.02026/) (2019)
- [Fast Symmetric Diffeomorphic Image Registration with Convolutional Neural Networks](https://mlsanity.com/paper/2003.09514/) (2020)
- [Weakly-Supervised Convolutional Neural Networks for Multimodal Image Registration](https://mlsanity.com/paper/1807.03361/) (2018)
- [DeepAtlas: Joint Semi-Supervised Learning of Image Registration and Segmentation](https://mlsanity.com/paper/1904.08465/) (2019)
- [Learning a Probabilistic Model for Diffeomorphic Registration](https://mlsanity.com/paper/1812.07460/) (2018)
- [DiffuseMorph: Unsupervised Deformable Image Registration Using Diffusion Model](https://mlsanity.com/paper/2112.05149/) (2021)
- [XMorpher: Full Transformer for Deformable Medical Image Registration via Cross Attention](https://mlsanity.com/paper/2206.07349/) (2022)
- [U-Net vs Transformer: Is U-Net Outdated in Medical Image Registration?](https://mlsanity.com/paper/2208.04939/) (2022)
- [DeepFLASH: An Efficient Network for Learning-based Medical Image Registration](https://mlsanity.com/paper/2004.02097/) (2020)
- [Image-and-Spatial Transformer Networks for Structure-Guided Image Registration](https://mlsanity.com/paper/1907.09200/) (2019)
- [Cross-modal Attention for MRI and Ultrasound Volume Registration](https://mlsanity.com/paper/2107.04548/) (2021)
- [Non-iterative Coarse-to-fine Registration based on Single-pass Deep Cumulative Learning](https://mlsanity.com/paper/2206.12596/) (2022)
- [Biomedical image analysis competitions: The state of current participation practice](https://mlsanity.com/paper/2212.08568/) (2022)
- [FAIM -- A ConvNet Method for Unsupervised 3D Medical Image Registration](https://mlsanity.com/paper/1811.09243/) (2018)
- [Deep Complementary Joint Model for Complex Scene Registration and Few-shot Segmentation on Medical Images](https://mlsanity.com/paper/2008.00710/) (2020)
- [Enhancing Medical Image Registration via Appearance Adjustment Networks](https://mlsanity.com/paper/2103.05213/) (2021)
- [Brain Tumor Sequence Registration with Non-iterative Coarse-to-fine Networks and Dual Deep Supervision](https://mlsanity.com/paper/2211.07876/) (2022)
- [Joint Registration and Segmentation via Multi-Task Learning for Adaptive Radiotherapy of Prostate Cancer](https://mlsanity.com/paper/2105.01844/) (2021)
- [High-quality Panorama Stitching based on Asymmetric Bidirectional Optical Flow](https://mlsanity.com/paper/2006.01201/) (2020)
- [Deep Morphological Simplification Network (MS-Net) for Guided Registration of Brain Magnetic Resonance Images](https://mlsanity.com/paper/1902.02342/) (2019)

## Built on by (most cited first, 6 of 6)

- [Correlation-aware Coarse-to-fine MLPs for Deformable Medical Image Registration](https://mlsanity.com/paper/2406.00123/) (2024)
- [Full-resolution MLPs Empower Medical Dense Prediction](https://mlsanity.com/paper/2311.16707/) (2023)
- [Dual-Attention Frequency Fusion at Multi-Scale for Joint Segmentation and Deformable Medical Image Registration](https://mlsanity.com/paper/2409.19658/) (2024)
- [Dynamic PET Image Prediction Using a Network Combining Reversible and Irreversible Modules](https://mlsanity.com/paper/2410.22674/) (2024)
- [Toward Next-generation Medical Vision Backbones: Modeling Finer-grained Long-range Visual Dependency](https://mlsanity.com/paper/2509.11328/) (2025)
- [Interpretable Unsupervised Deformable Image Registration via Confidence-bound Multi-Hop Visual Reasoning](https://mlsanity.com/paper/2602.00211/) (2026)

## Abstract (arXiv)

Deformable image registration aims to find a dense non-linear spatial correspondence between a pair of images, which is a crucial step for many medical tasks such as tumor growth monitoring and population analysis. Recently, Deep Neural Networks (DNNs) have been widely recognized for their ability to perform fast end-to-end registration. However, DNN-based registration needs to explore the spatial information of each image and fuse this information to characterize spatial correspondence. This raises an essential question: what is the optimal fusion strategy to characterize spatial correspondence? Existing fusion strategies (e.g., early fusion, late fusion) were empirically designed to fuse information by manually defined prior knowledge, which inevitably constrains the registration performance within the limits of empirical designs. In this study, we depart from existing empirically-designed fusion strategies and develop a data-driven fusion strategy for deformable image registration. To achieve this, we propose an Automatic Fusion network (AutoFuse) that provides flexibility to fuse information at many potential locations within the network. A Fusion Gate (FG) module is also proposed to control how to fuse information at each potential network location based on training data. Our AutoFuse can automatically optimize its fusion strategy during training and can be generalizable to both unsupervised registration (without any labels) and semi-supervised registration (with weak labels provided for partial training data). Extensive experiments on two well-benchmarked medical registration tasks (inter- and intra-patient registration) with eight public datasets show that our AutoFuse outperforms state-of-the-art unsupervised and semi-supervised registration methods.