# Learning the Effect of Registration Hyperparameters with HyperMorph

arXiv: https://arxiv.org/abs/2203.16680 · HTML: https://mlsanity.com/paper/2203.16680/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 91 references that are in the map.
- First posted on arXiv 30 Mar 2022.

## Builds on (most cited first, 30 of 40)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [HyperNetworks](https://mlsanity.com/paper/1609.09106/) (2016)
- [Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization](https://mlsanity.com/paper/1603.06560/) (2016)
- [TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems](https://mlsanity.com/paper/1603.04467/) (2016)
- [VoxelMorph: A Learning Framework for Deformable Medical Image Registration](https://mlsanity.com/paper/1809.05231/) (2018)
- [Bilevel Programming for Hyperparameter Optimization and Meta-Learning](https://mlsanity.com/paper/1806.04910/) (2018)
- [Gradient-based Hyperparameter Optimization through Reversible Learning](https://mlsanity.com/paper/1502.03492/) (2015)
- [A Learned Representation For Artistic Style](https://mlsanity.com/paper/1610.07629/) (2016)
- [A Probabilistic U-Net for Segmentation of Ambiguous Images](https://mlsanity.com/paper/1806.05034/) (2018)
- [Forward and Reverse Gradient-Based Hyperparameter Optimization](https://mlsanity.com/paper/1703.01785/) (2017)
- [Hyperparameter optimization with approximate gradient](https://mlsanity.com/paper/1602.02355/) (2016)
- [MetaPruning: Meta Learning for Automatic Neural Network Channel Pruning](https://mlsanity.com/paper/1903.10258/) (2019)
- [Non-stochastic Best Arm Identification and Hyperparameter Optimization](https://mlsanity.com/paper/1502.07943/) (2015)
- [SMASH: One-Shot Model Architecture Search through HyperNetworks](https://mlsanity.com/paper/1708.05344/) (2017)
- [Unsupervised Learning of Probabilistic Diffeomorphic Registration for Images and Surfaces](https://mlsanity.com/paper/1903.03545/) (2019)
- [A Deep Learning Framework for Unsupervised Affine and Deformable Image Registration](https://mlsanity.com/paper/1809.06130/) (2018)
- [Large Deformation Diffeomorphic Image Registration with Laplacian Pyramid Networks](https://mlsanity.com/paper/2006.16148/) (2020)
- [Multi-fidelity Bayesian Optimisation with Continuous Approximations](https://mlsanity.com/paper/1703.06240/) (2017)
- [Quicksilver: Fast Predictive Image Registration - a Deep Learning Approach](https://mlsanity.com/paper/1703.10908/) (2017)
- [Self-Tuning Networks: Bilevel Optimization of Hyperparameters using Structured Best-Response Functions](https://mlsanity.com/paper/1903.03088/) (2019)
- [Recursive Cascaded Networks for Unsupervised Medical Image Registration](https://mlsanity.com/paper/1907.12353/) (2019)
- [Weakly-Supervised Convolutional Neural Networks for Multimodal Image Registration](https://mlsanity.com/paper/1807.03361/) (2018)
- [HyperMorph: Amortized Hyperparameter Learning for Image Registration](https://mlsanity.com/paper/2101.01035/) (2021)
- [Stochastic Hyperparameter Optimization through Hypernetworks](https://mlsanity.com/paper/1802.09419/) (2018)
- [Scalable Gradient-Based Tuning of Continuous Regularization Hyperparameters](https://mlsanity.com/paper/1511.06727/) (2015)
- [Bayesian Hypernetworks](https://mlsanity.com/paper/1710.04759/) (2017)
- [Hypernetwork functional image representation](https://mlsanity.com/paper/1902.10404/) (2019)
- [Learning Conditional Deformable Templates with Convolutional Networks](https://mlsanity.com/paper/1908.02738/) (2019)

## Built on by (most cited first, 30 of 35)

- [UniverSeg: Universal Medical Image Segmentation](https://mlsanity.com/paper/2304.06131/) (2023)
- [A survey on deep learning in medical image registration: new technologies, uncertainty, evaluation metrics, and beyond](https://mlsanity.com/paper/2307.15615/) (2023)
- [ScribblePrompt: Fast and Flexible Interactive Segmentation for Any Biomedical Image](https://mlsanity.com/paper/2312.07381/) (2023)
- [Learning General-Purpose Biomedical Volume Representations using Randomized Synthesis](https://mlsanity.com/paper/2411.02372/) (2024)
- [Neuralizer: General Neuroimage Analysis without Re-Training](https://mlsanity.com/paper/2305.02644/) (2023)
- [Tyche: Stochastic In-Context Learning for Medical Image Segmentation](https://mlsanity.com/paper/2401.13650/) (2024)
- [Anatomy-aware and acquisition-agnostic joint registration with SynthMorph](https://mlsanity.com/paper/2301.11329/) (2023)
- [Magnitude Invariant Parametrizations Improve Hypernetwork Learning](https://mlsanity.com/paper/2304.07645/) (2023)
- [Learning Transferable Features for Implicit Neural Representations](https://mlsanity.com/paper/2409.09566/) (2024)
- [Conformal Prediction for Image Segmentation Using Morphological Prediction Sets](https://mlsanity.com/paper/2503.05618/) (2025)
- [MultiverSeg: Scalable Interactive Segmentation of Biomedical Imaging Datasets with In-Context Guidance](https://mlsanity.com/paper/2412.15058/) (2024)
- [Inverse Consistency by Construction for Multistep Deep Registration](https://mlsanity.com/paper/2305.00087/) (2023)
- [Fit Pixels, Get Labels: Meta-learned Implicit Networks for Image Segmentation](https://mlsanity.com/paper/2510.04021/) (2025)
- [Registration by Regression (RbR): a framework for interpretable and flexible atlas registration](https://mlsanity.com/paper/2404.16781/) (2024)
- [Global-Local Image Perceptual Score (GLIPS): Evaluating Photorealistic Quality of AI-Generated Images](https://mlsanity.com/paper/2405.09426/) (2024)
- [HyperSpace: Hypernetworks for spacing-adaptive image segmentation](https://mlsanity.com/paper/2407.03681/) (2024)
- [Guiding Registration with Emergent Similarity from Pre-Trained Diffusion Models](https://mlsanity.com/paper/2506.02419/) (2025)
- [Decoder-Only Image Registration](https://mlsanity.com/paper/2402.03585/) (2024)
- [Towards Saner Deep Image Registration](https://mlsanity.com/paper/2307.09696/) (2023)
- [Capturing Longitudinal Changes in Brain Morphology Using Temporally Parameterized Neural Displacement Fields](https://mlsanity.com/paper/2504.09514/) (2025)
- [Medverse: A Universal Model for Full-Resolution 3D Medical Image Segmentation, Transformation and Enhancement](https://mlsanity.com/paper/2509.09232/) (2025)
- [Atlas is Your Perfect Context: One-Shot Customization for Generalizable Foundational Medical Image Segmentation](https://mlsanity.com/paper/2512.18176/) (2025)
- [Gaussian Primitives for Deformable Image Registration](https://mlsanity.com/paper/2406.03394/) (2024)
- [Unsupervised learning of spatially varying regularization for diffeomorphic image registration](https://mlsanity.com/paper/2412.17982/) (2024)
- [Distillation Learning Guided by Image Reconstruction for One-Shot Medical Image Segmentation](https://mlsanity.com/paper/2408.03616/) (2024)
- [Neuroverse3D: Developing In-Context Learning Universal Model for Neuroimaging in 3D](https://mlsanity.com/paper/2503.02410/) (2025)
- [Evaluation of Deformable Image Registration under Alignment-Regularity Trade-off](https://mlsanity.com/paper/2503.07185/) (2025)
- [Scale-Space Hypernetworks for Efficient Biomedical Imaging](https://mlsanity.com/paper/2304.05448/) (2023)
- [A Symmetric Dynamic Learning Framework for Diffeomorphic Medical Image Registration](https://mlsanity.com/paper/2411.02888/) (2024)
- [MUSTER: Longitudinal Deformable Registration by Composition of Consecutive Deformations](https://mlsanity.com/paper/2412.14671/) (2024)

## Abstract (arXiv)

We introduce HyperMorph, a framework that facilitates efficient hyperparameter tuning in learning-based deformable image registration. Classical registration algorithms perform an iterative pair-wise optimization to compute a deformation field that aligns two images. Recent learning-based approaches leverage large image datasets to learn a function that rapidly estimates a deformation for a given image pair. In both strategies, the accuracy of the resulting spatial correspondences is strongly influenced by the choice of certain hyperparameter values. However, an effective hyperparameter search consumes substantial time and human effort as it often involves training multiple models for different fixed hyperparameter values and may lead to suboptimal registration. We propose an amortized hyperparameter learning strategy to alleviate this burden by learning the impact of hyperparameters on deformation fields. We design a meta network, or hypernetwork, that predicts the parameters of a registration network for input hyperparameters, thereby comprising a single model that generates the optimal deformation field corresponding to given hyperparameter values. This strategy enables fast, high-resolution hyperparameter search at test-time, reducing the inefficiency of traditional approaches while increasing flexibility. We also demonstrate additional benefits of HyperMorph, including enhanced robustness to model initialization and the ability to rapidly identify optimal hyperparameter values specific to a dataset, image contrast, task, or even anatomical region, all without the need to retrain models. We make our code publicly available at http://hypermorph.voxelmorph.net.