# Multi-scale Neural ODEs for 3D Medical Image Registration

arXiv: https://arxiv.org/abs/2106.08493 · HTML: https://mlsanity.com/paper/2106.08493/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 33 references that are in the map.
- First posted on arXiv 16 Jun 2021.

## Builds on (most cited first, 17 of 17)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Multimodal Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/1804.04732/) (2018)
- [VoxelMorph: A Learning Framework for Deformable Medical Image Registration](https://mlsanity.com/paper/1809.05231/) (2018)
- [Wasserstein GAN](https://mlsanity.com/paper/1701.07875/) (2017)
- [DRIT++: Diverse Image-to-Image Translation via Disentangled Representations](https://mlsanity.com/paper/1905.01270/) (2019)
- [A Deep Learning Framework for Unsupervised Affine and Deformable Image Registration](https://mlsanity.com/paper/1809.06130/) (2018)
- [Quicksilver: Fast Predictive Image Registration - a Deep Learning Approach](https://mlsanity.com/paper/1703.10908/) (2017)
- [Recursive Cascaded Networks for Unsupervised Medical Image Registration](https://mlsanity.com/paper/1907.12353/) (2019)
- [Adaptive Checkpoint Adjoint Method for Gradient Estimation in Neural ODE](https://mlsanity.com/paper/2006.02493/) (2020)
- [End-to-End Unsupervised Deformable Image Registration with a Convolutional Neural Network](https://mlsanity.com/paper/1704.06065/) (2017)
- [Unsupervised Multi-Modal Image Registration via Geometry Preserving Image-to-Image Translation](https://mlsanity.com/paper/2003.08073/) (2020)
- [Unsupervised Deformable Registration for Multi-Modal Images via Disentangled Representations](https://mlsanity.com/paper/1903.09331/) (2019)
- [Networks for Joint Affine and Non-parametric Image Registration](https://mlsanity.com/paper/1903.08811/) (2019)
- [MRI Image Reconstruction via Learning Optimization Using Neural ODEs](https://mlsanity.com/paper/2006.13825/) (2020)
- [Robust Multimodal Image Registration Using Deep Recurrent Reinforcement Learning](https://mlsanity.com/paper/2002.03733/) (2020)

## Built on by (most cited first, 18 of 18)

- [A survey on deep learning in medical image registration: new technologies, uncertainty, evaluation metrics, and beyond](https://mlsanity.com/paper/2307.15615/) (2023)
- [NODEO: A Neural Ordinary Differential Equation Based Optimization Framework for Deformable Image Registration](https://mlsanity.com/paper/2108.03443/) (2021)
- [Diffeomorphic Image Registration with Neural Velocity Field](https://mlsanity.com/paper/2202.12498/) (2022)
- [Medical Image Registration via Neural Fields](https://mlsanity.com/paper/2206.03111/) (2022)
- [Embedding Gradient-based Optimization in Image Registration Networks](https://mlsanity.com/paper/2112.03915/) (2021)
- [TO-FLOW: Efficient Continuous Normalizing Flows with Temporal Optimization adjoint with Moving Speed](https://mlsanity.com/paper/2203.10335/) (2022)
- [Physics-Informed Machine Learning in Biomedical Science and Engineering](https://mlsanity.com/paper/2510.05433/) (2025)
- [Disentangling Progress in Medical Image Registration: Beyond Trend-Driven Architectures towards Domain-Specific Strategies](https://mlsanity.com/paper/2512.01913/) (2025)
- [ORRN: An ODE-based Recursive Registration Network for Deformable Respiratory Motion Estimation with Lung 4DCT Images](https://mlsanity.com/paper/2305.14673/) (2023)
- [CPT-Interp: Continuous sPatial and Temporal Motion Modeling for 4D Medical Image Interpolation](https://mlsanity.com/paper/2405.15385/) (2024)
- [Echocardiography Segmentation Using Neural ODE-based Diffeomorphic Registration Field](https://mlsanity.com/paper/2306.09687/) (2023)
- [Recurrent Inference Machine for Medical Image Registration](https://mlsanity.com/paper/2406.13413/) (2024)
- [Modality-Aware Feature Matching in Visual and Vision-Language Applications: A Comprehensive Survey](https://mlsanity.com/paper/2507.22791/) (2025)
- [Collaborative Quantization Embeddings for Intra-Subject Prostate MR Image Registration](https://mlsanity.com/paper/2207.06189/) (2022)
- [Learning Diffeomorphism for Image Registration with Time-Continuous Networks using Semigroup Regularization](https://mlsanity.com/paper/2405.18684/) (2024)
- [Continuous sPatial-Temporal Deformable Image Registration (CPT-DIR) for motion modelling in radiotherapy: beyond classic voxel-based methods](https://mlsanity.com/paper/2405.00430/) (2024)
- [Multimodal Diffeomorphic Registration with Neural ODEs and Structural Descriptors](https://mlsanity.com/paper/2512.22689/) (2025)
- [Two-Parameter Flow Map Learning for Continuous-Time Diffeomorphic Image Registration](https://mlsanity.com/paper/2609.10789/) (2026)

## Abstract (arXiv)

Image registration plays an important role in medical image analysis. Conventional optimization based methods provide an accurate estimation due to the iterative process at the cost of expensive computation. Deep learning methods such as learn-to-map are much faster but either iterative or coarse-to-fine approach is required to improve accuracy for handling large motions. In this work, we proposed to learn a registration optimizer via a multi-scale neural ODE model. The inference consists of iterative gradient updates similar to a conventional gradient descent optimizer but in a much faster way, because the neural ODE learns from the training data to adapt the gradient efficiently at each iteration. Furthermore, we proposed to learn a modal-independent similarity metric to address image appearance variations across different image contrasts. We performed evaluations through extensive experiments in the context of multi-contrast 3D MR images from both public and private data sources and demonstrate the superior performance of our proposed methods.