# Anatomy-aware and acquisition-agnostic joint registration with SynthMorph

arXiv: https://arxiv.org/abs/2301.11329 · HTML: https://mlsanity.com/paper/2301.11329/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 47 of its 170 references that are in the map.
- First posted on arXiv 26 Jan 2023.

## Builds on (most cited first, 30 of 47)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [A Comprehensive Survey on Transfer Learning](https://mlsanity.com/paper/1911.02685/) (2019)
- [VoxelMorph: A Learning Framework for Deformable Medical Image Registration](https://mlsanity.com/paper/1809.05231/) (2018)
- [Deep Visual Domain Adaptation: A Survey](https://mlsanity.com/paper/1802.03601/) (2018)
- [Return of Frustratingly Easy Domain Adaptation](https://mlsanity.com/paper/1511.05547/) (2015)
- [The Effectiveness of Data Augmentation in Image Classification using Deep Learning](https://mlsanity.com/paper/1712.04621/) (2017)
- [SynthSeg: Segmentation of brain MRI scans of any contrast and resolution without retraining](https://mlsanity.com/paper/2107.09559/) (2021)
- [Unsupervised Learning of Probabilistic Diffeomorphic Registration for Images and Surfaces](https://mlsanity.com/paper/1903.03545/) (2019)
- [A Deep Learning Framework for Unsupervised Affine and Deformable Image Registration](https://mlsanity.com/paper/1809.06130/) (2018)
- [More ConvNets in the 2020s: Scaling up Kernels Beyond 51x51 using Sparsity](https://mlsanity.com/paper/2207.03620/) (2022)
- [SynthMorph: learning contrast-invariant registration without acquired images](https://mlsanity.com/paper/2004.10282/) (2020)
- [Data augmentation using learned transformations for one-shot medical image segmentation](https://mlsanity.com/paper/1902.09383/) (2019)
- [SynthStrip: Skull-Stripping for Any Brain Image](https://mlsanity.com/paper/2203.09974/) (2022)
- [Unsupervised domain adaptation in brain lesion segmentation with adversarial networks](https://mlsanity.com/paper/1612.08894/) (2016)
- [Unsupervised Learning for Fast Probabilistic Diffeomorphic Registration](https://mlsanity.com/paper/1805.04605/) (2018)
- [Quicksilver: Fast Predictive Image Registration - a Deep Learning Approach](https://mlsanity.com/paper/1703.10908/) (2017)
- [Recursive Cascaded Networks for Unsupervised Medical Image Registration](https://mlsanity.com/paper/1907.12353/) (2019)
- [Deep Learning in Medical Image Registration: A Review](https://mlsanity.com/paper/1912.12318/) (2019)
- [Deep Visual Domain Adaptation](https://mlsanity.com/paper/2012.14176/) (2020)
- [HyperMorph: Amortized Hyperparameter Learning for Image Registration](https://mlsanity.com/paper/2101.01035/) (2021)
- [End-to-End Unsupervised Deformable Image Registration with a Convolutional Neural Network](https://mlsanity.com/paper/1704.06065/) (2017)
- [Learning Conditional Deformable Templates with Convolutional Networks](https://mlsanity.com/paper/1908.02738/) (2019)
- [Learning a Probabilistic Model for Diffeomorphic Registration](https://mlsanity.com/paper/1812.07460/) (2018)
- [Unsupervised 3D End-to-End Medical Image Registration with Volume Tweening Network](https://mlsanity.com/paper/1902.05020/) (2019)
- [Unsupervised Deformable Registration for Multi-Modal Images via Disentangled Representations](https://mlsanity.com/paper/1903.09331/) (2019)
- [Networks for Joint Affine and Non-parametric Image Registration](https://mlsanity.com/paper/1903.08811/) (2019)

## Built on by (most cited first, 19 of 19)

- [MultiMorph: On-demand Atlas Construction](https://mlsanity.com/paper/2504.00247/) (2025)
- [Boosting Skull-Stripping Performance for Pediatric Brain Images](https://mlsanity.com/paper/2402.16634/) (2024)
- [Registration by Regression (RbR): a framework for interpretable and flexible atlas registration](https://mlsanity.com/paper/2404.16781/) (2024)
- [VoxelPrompt: A Vision Agent for End-to-End Medical Image Analysis](https://mlsanity.com/paper/2410.08397/) (2024)
- [Domain-randomized deep learning for neuroimage analysis](https://mlsanity.com/paper/2507.13458/) (2025)
- [Synthetic Data for Robust Stroke Segmentation](https://mlsanity.com/paper/2404.01946/) (2024)
- [Learning accurate rigid registration for longitudinal brain MRI from synthetic data](https://mlsanity.com/paper/2501.13010/) (2025)
- [Preserving Tumor Volumes for Unsupervised Medical Image Registration](https://mlsanity.com/paper/2309.10153/) (2023)
- [TimeFlow: Temporal Conditioning for Longitudinal Brain MRI Registration and Aging Analysis](https://mlsanity.com/paper/2501.08667/) (2025)
- [Coarse-to-Fine Joint Registration of MR and Ultrasound Images via Imaging Style Transfer](https://mlsanity.com/paper/2508.05240/) (2025)
- [Deep infant brain segmentation from multi-contrast MRI](https://mlsanity.com/paper/2512.05114/) (2025)
- [BTReport: A Framework for Brain Tumor Radiology Report Generation with Clinically Relevant Features](https://mlsanity.com/paper/2602.16006/) (2026)
- [Impact of Initialization on Intra-subject Pediatric Brain MR Image Registration: A Comparative Analysis between SyN ANTs and Deep Learning-Based Approaches](https://mlsanity.com/paper/2406.19943/) (2024)
- [FetDTIAlign: A Deep Learning Framework for Affine and Deformable Registration of Fetal Brain dMRI](https://mlsanity.com/paper/2502.01057/) (2025)
- [Unified Brain Surface and Volume Registration](https://mlsanity.com/paper/2512.19928/) (2025)
- [Why Invariance is Not Enough for Biomedical Domain Generalization and How to Fix It](https://mlsanity.com/paper/2604.02564/) (2026)
- [Patient-Agnostic Synthetic Pretraining for Efficient Patient-Specific Intraoperative 2D/3D Registration](https://mlsanity.com/paper/2607.23343/) (2026)
- [textttDisMorph: learning to disentangle technical distortions from true biological change](https://mlsanity.com/paper/2608.08173/) (2026)
- [Synthetic training for long-tail haemorrhagic lesion segmentation in data-scarce settings](https://mlsanity.com/paper/2610.01542/) (2026)

## Abstract (arXiv)

Affine image registration is a cornerstone of medical image analysis. While classical algorithms can achieve excellent accuracy, they solve a time-consuming optimization for every image pair. Deep-learning (DL) methods learn a function that maps an image pair to an output transform. Evaluating the function is fast, but capturing large transforms can be challenging, and networks tend to struggle if a test-image characteristic shifts from the training domain, such as resolution. Most affine methods are agnostic to the anatomy the user wishes to align, meaning the registration will be inaccurate if algorithms consider all structures in the image. We address these shortcomings with SynthMorph, a fast, symmetric, diffeomorphic, and easy-to-use DL tool for joint affine-deformable registration of any brain image without preprocessing. First, we leverage a strategy that trains networks with widely varying images synthesized from label maps, yielding robust performance for image types unseen at training. Second, we optimize the spatial overlap of select anatomical labels. This enables networks to distinguish anatomy of interest from irrelevant structures, removing the need for preprocessing that excludes content that may impinge on anatomy-specific registration. Third, we combine the affine model with a deformable hypernetwork that lets users choose the optimal deformation-field regularity for their specific data, at registration time, in a fraction of the time required by classical methods. We analyze how competing architectures learn affine transforms and compare state-of-the-art registration tools across an extremely diverse set of neuroimaging data, aiming to truly capture the behavior of methods in the real world. SynthMorph demonstrates high accuracy and is available at https://w3id.org/synthmorph, as a single complete end-to-end solution for registration of brain MRI.