# Voxelmorph++ Going beyond the cranial vault with keypoint supervision and multi-channel instance optimisation

arXiv: https://arxiv.org/abs/2203.00046 · HTML: https://mlsanity.com/paper/2203.00046/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 29 references that are in the map.
- First posted on arXiv 28 Feb 2022.

## Builds on (most cited first, 15 of 15)

- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [VoxelMorph: A Learning Framework for Deformable Medical Image Registration](https://mlsanity.com/paper/1809.05231/) (2018)
- [FlowNet3D: Learning Scene Flow in 3D Point Clouds](https://mlsanity.com/paper/1806.01411/) (2018)
- [A Deep Learning Framework for Unsupervised Affine and Deformable Image Registration](https://mlsanity.com/paper/1809.06130/) (2018)
- [Large Deformation Diffeomorphic Image Registration with Laplacian Pyramid Networks](https://mlsanity.com/paper/2006.16148/) (2020)
- [Learn2Reg: comprehensive multi-task medical image registration challenge, dataset and evaluation in the era of deep learning](https://mlsanity.com/paper/2112.04489/) (2021)
- [Unsupervised 3D End-to-End Medical Image Registration with Volume Tweening Network](https://mlsanity.com/paper/1902.05020/) (2019)
- [Conditional Deformable Image Registration with Convolutional Neural Network](https://mlsanity.com/paper/2106.12673/) (2021)
- [Dual-Stream Pyramid Registration Network](https://mlsanity.com/paper/1909.11966/) (2019)
- [Closing the Gap between Deep and Conventional Image Registration using Probabilistic Dense Displacement Networks](https://mlsanity.com/paper/1907.10931/) (2019)
- [Fast 3D registration with accurate optimisation and little learning for Learn2Reg 2021](https://mlsanity.com/paper/2112.03053/) (2021)
- [CNN-based Lung CT Registration with Multiple Anatomical Constraints](https://mlsanity.com/paper/2011.14372/) (2020)
- [Learning Deformable Point Set Registration with Regularized Dynamic Graph CNNs for Large Lung Motion in COPD Patients](https://mlsanity.com/paper/1909.07818/) (2019)
- [MICS : Multi-steps, Inverse Consistency and Symmetric deep learning registration network](https://mlsanity.com/paper/2111.12123/) (2021)

## Built on by (most cited first, 19 of 19)

- [A survey on deep learning in medical image registration: new technologies, uncertainty, evaluation metrics, and beyond](https://mlsanity.com/paper/2307.15615/) (2023)
- [General Purpose Image Encoder DINOv2 for Medical Image Registration](https://mlsanity.com/paper/2402.15687/) (2024)
- [Vector Field Attention for Deformable Image Registration](https://mlsanity.com/paper/2407.10209/) (2024)
- [Adapting the Mean Teacher for keypoint-based lung registration under geometric domain shifts](https://mlsanity.com/paper/2207.00371/) (2022)
- [Deep Implicit Optimization enables Robust Learnable Features for Deformable Image Registration](https://mlsanity.com/paper/2406.07361/) (2024)
- [GuidedMorph: Two-Stage Deformable Registration for Breast MRI](https://mlsanity.com/paper/2505.13414/) (2025)
- [Distance Map Supervised Landmark Localization for MR-TRUS Registration](https://mlsanity.com/paper/2210.05738/) (2022)
- [Recurrence With Correlation Network for Medical Image Registration](https://mlsanity.com/paper/2302.02283/) (2023)
- [Implicit Neural Representations for Registration of Left Ventricle Myocardium During a Cardiac Cycle](https://mlsanity.com/paper/2501.07248/) (2025)
- [Mono-Modalizing Extremely Heterogeneous Multi-Modal Medical Image Registration](https://mlsanity.com/paper/2506.15596/) (2025)
- [Deep Biomechanically-Guided Interpolation for Keypoint-Based Brain Shift Registration](https://mlsanity.com/paper/2508.13762/) (2025)
- [Gaussian Primitive Optimized Deformable Retinal Image Registration](https://mlsanity.com/paper/2508.16852/) (2025)
- [CT respiratory motion synthesis using joint supervised and adversarial learning](https://mlsanity.com/paper/2404.00163/) (2024)
- [OncoReg: Medical Image Registration for Oncological Challenges](https://mlsanity.com/paper/2503.23179/) (2025)
- [Ideal Registration? Segmentation is All You Need](https://mlsanity.com/paper/2509.15784/) (2025)
- [Data-driven registration and modeling of brain deformation for image-guided neurosurgery](https://mlsanity.com/paper/2602.10155/) (2026)
- [Foundation Models for Medical Imaging: Status, Challenges, and Directions](https://mlsanity.com/paper/2602.15913/) (2026)
- [GLIDE-Reg: Global-to-Local Deformable Registration Using Co-Optimized Foundation and Handcrafted Features](https://mlsanity.com/paper/2603.00218/) (2026)
- [An Accessible Solution for Deformable Image Registration Compared with Learning-Based Approaches](https://mlsanity.com/paper/2608.02248/) (2026)

## Abstract (arXiv)

The majority of current research in deep learning based image registration addresses inter-patient brain registration with moderate deformation magnitudes. The recent Learn2Reg medical registration benchmark has demonstrated that single-scale U-Net architectures, such as VoxelMorph that directly employ a spatial transformer loss, often do not generalise well beyond the cranial vault and fall short of state-of-the-art performance for abdominal or intra-patient lung registration. Here, we propose two straightforward steps that greatly reduce this gap in accuracy. First, we employ keypoint self-supervision with a novel network head that predicts a discretised heatmap and robustly reduces large deformations for better robustness. Second, we replace multiple learned fine-tuning steps by a single instance optimisation with hand-crafted features and the Adam optimiser. Different to other related work, including FlowNet or PDD-Net, our approach does not require a fully discretised architecture with correlation layer. Our ablation study demonstrates the importance of keypoints in both self-supervised and unsupervised (using only a MIND metric) settings. On a multi-centric inspiration-exhale lung CT dataset, including very challenging COPD scans, our method outperforms VoxelMorph by improving nonlinear alignment by 77% compared to 19% - reaching target registration errors of 2 mm that outperform all but one learning methods published to date. Extending the method to semantic features sets new stat-of-the-art performance on inter-subject abdominal CT registration.