# Test-Time Training for Deformable Multi-Scale Image Registration

arXiv: https://arxiv.org/abs/2103.13578 · HTML: https://mlsanity.com/paper/2103.13578/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 50 references that are in the map.
- First posted on arXiv 25 Mar 2021.

## Builds on (most cited first, 26 of 26)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [Unsupervised Monocular Depth Estimation with Left-Right Consistency](https://mlsanity.com/paper/1609.03677/) (2016)
- [VoxelMorph: A Learning Framework for Deformable Medical Image Registration](https://mlsanity.com/paper/1809.05231/) (2018)
- [A large annotated medical image dataset for the development and evaluation of segmentation algorithms](https://mlsanity.com/paper/1902.09063/) (2019)
- [A Deep Learning Framework for Unsupervised Affine and Deformable Image Registration](https://mlsanity.com/paper/1809.06130/) (2018)
- [An Unsupervised Learning Model for Deformable Medical Image Registration](https://mlsanity.com/paper/1802.02604/) (2018)
- [Large Deformation Diffeomorphic Image Registration with Laplacian Pyramid Networks](https://mlsanity.com/paper/2006.16148/) (2020)
- [Iterative Residual Refinement for Joint Optical Flow and Occlusion Estimation](https://mlsanity.com/paper/1904.05290/) (2019)
- [Deep Learning in Medical Image Registration: A Survey](https://mlsanity.com/paper/1903.02026/) (2019)
- [Unsupervised Learning for Fast Probabilistic Diffeomorphic Registration](https://mlsanity.com/paper/1805.04605/) (2018)
- [Quicksilver: Fast Predictive Image Registration - a Deep Learning Approach](https://mlsanity.com/paper/1703.10908/) (2017)
- [Recursive Cascaded Networks for Unsupervised Medical Image Registration](https://mlsanity.com/paper/1907.12353/) (2019)
- [End-to-End Unsupervised Deformable Image Registration with a Convolutional Neural Network](https://mlsanity.com/paper/1704.06065/) (2017)
- [Test-Time Training with Self-Supervision for Generalization under Distribution Shifts](https://mlsanity.com/paper/1909.13231/) (2019)
- [AnatomyNet: Deep Learning for Fast and Fully Automated Whole-volume Segmentation of Head and Neck Anatomy](https://mlsanity.com/paper/1808.05238/) (2018)
- [Multi-Domain Image Completion for Random Missing Input Data](https://mlsanity.com/paper/2007.05534/) (2020)
- [Non-Rigid Image Registration Using Self-Supervised Fully Convolutional Networks without Training Data](https://mlsanity.com/paper/1801.04012/) (2018)
- [An Artificial Agent for Robust Image Registration](https://mlsanity.com/paper/1611.10336/) (2016)
- [Dilated FCN for Multi-Agent 2D/3D Medical Image Registration](https://mlsanity.com/paper/1712.01651/) (2017)
- [Cycle-Consistent Adversarial Autoencoders for Unsupervised Text Style Transfer](https://mlsanity.com/paper/2010.00735/) (2020)
- [AIRNet: Self-Supervised Affine Registration for 3D Medical Images using Neural Networks](https://mlsanity.com/paper/1810.02583/) (2018)
- [LAMP: Large Deep Nets with Automated Model Parallelism for Image Segmentation](https://mlsanity.com/paper/2006.12575/) (2020)
- [Adversarial Deep Structured Nets for Mass Segmentation from Mammograms](https://mlsanity.com/paper/1710.09288/) (2017)
- [NeurReg: Neural Registration and Its Application to Image Segmentation](https://mlsanity.com/paper/1910.01763/) (2019)

## Built on by (most cited first, 17 of 17)

- [A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts](https://mlsanity.com/paper/2303.15361/) (2023)
- [Self-Supervised Monocular Depth and Ego-Motion Estimation in Endoscopy: Appearance Flow to the Rescue](https://mlsanity.com/paper/2112.08122/) (2021)
- [DLTTA: Dynamic Learning Rate for Test-time Adaptation on Cross-domain Medical Images](https://mlsanity.com/paper/2205.13723/) (2022)
- [Diffeomorphic Image Registration with Neural Velocity Field](https://mlsanity.com/paper/2202.12498/) (2022)
- [Medical Image Registration via Neural Fields](https://mlsanity.com/paper/2206.03111/) (2022)
- [Towards Open-World Segmentation of Parts](https://mlsanity.com/paper/2305.16804/) (2023)
- [Robust deformable image registration using cycle-consistent implicit representations](https://mlsanity.com/paper/2310.01934/) (2023)
- [Meta-Learning Initializations for Interactive Medical Image Registration](https://mlsanity.com/paper/2210.15371/) (2022)
- [Meta-Registration: Learning Test-Time Optimization for Single-Pair Image Registration](https://mlsanity.com/paper/2207.10996/) (2022)
- [Adversarial Contrastive Self-Supervised Learning](https://mlsanity.com/paper/2202.13072/) (2022)
- [Echocardiography Segmentation Using Neural ODE-based Diffeomorphic Registration Field](https://mlsanity.com/paper/2306.09687/) (2023)
- [Deep learning in medical image registration: introduction and survey](https://mlsanity.com/paper/2309.00727/) (2023)
- [Enhanced Model Robustness to Input Corruptions by Per-corruption Adaptation of Normalization Statistics](https://mlsanity.com/paper/2407.06450/) (2024)
- [Region Specific Optimization (RSO)-based Deep Interactive Registration](https://mlsanity.com/paper/2203.04295/) (2022)
- [Multi-scale, Data-driven and Anatomically Constrained Deep Learning Image Registration for Adult and Fetal Echocardiography](https://mlsanity.com/paper/2309.00831/) (2023)
- [Med-PerSAM: One-Shot Visual Prompt Tuning for Personalized Segment Anything Model in Medical Domain](https://mlsanity.com/paper/2411.16123/) (2024)
- [Recursive Uncertainty-Gated Image Registration for Learning-based Algorithms](https://mlsanity.com/paper/2609.28081/) (2026)

## Abstract (arXiv)

Registration is a fundamental task in medical robotics and is often a crucial step for many downstream tasks such as motion analysis, intra-operative tracking and image segmentation. Popular registration methods such as ANTs and NiftyReg optimize objective functions for each pair of images from scratch, which are time-consuming for 3D and sequential images with complex deformations. Recently, deep learning-based registration approaches such as VoxelMorph have been emerging and achieve competitive performance. In this work, we construct a test-time training for deep deformable image registration to improve the generalization ability of conventional learning-based registration model. We design multi-scale deep networks to consecutively model the residual deformations, which is effective for high variational deformations. Extensive experiments validate the effectiveness of multi-scale deep registration with test-time training based on Dice coefficient for image segmentation and mean square error (MSE), normalized local cross-correlation (NLCC) for tissue dense tracking tasks. Two videos are in https://www.youtube.com/watch?v=NvLrCaqCiAE and https://www.youtube.com/watch?v=pEA6ZmtTNuQ