# multiGradICON: A Foundation Model for Multimodal Medical Image Registration

arXiv: https://arxiv.org/abs/2408.00221 · HTML: https://mlsanity.com/paper/2408.00221/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 58 references that are in the map.
- First posted on arXiv 1 Aug 2024.

## Builds on (most cited first, 24 of 24)

- [3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation](https://mlsanity.com/paper/1606.06650/) (2016)
- [VoxelMorph: A Learning Framework for Deformable Medical Image Registration](https://mlsanity.com/paper/1809.05231/) (2018)
- [SynthMorph: learning contrast-invariant registration without acquired images](https://mlsanity.com/paper/2004.10282/) (2020)
- [Large Deformation Diffeomorphic Image Registration with Laplacian Pyramid Networks](https://mlsanity.com/paper/2006.16148/) (2020)
- [Quicksilver: Fast Predictive Image Registration - a Deep Learning Approach](https://mlsanity.com/paper/1703.10908/) (2017)
- [Fast Symmetric Diffeomorphic Image Registration with Convolutional Neural Networks](https://mlsanity.com/paper/2003.09514/) (2020)
- [Weakly-Supervised Convolutional Neural Networks for Multimodal Image Registration](https://mlsanity.com/paper/1807.03361/) (2018)
- [HyperMorph: Amortized Hyperparameter Learning for Image Registration](https://mlsanity.com/paper/2101.01035/) (2021)
- [DeepAtlas: Joint Semi-Supervised Learning of Image Registration and Segmentation](https://mlsanity.com/paper/1904.08465/) (2019)
- [uniGradICON: A Foundation Model for Medical Image Registration](https://mlsanity.com/paper/2403.05780/) (2024)
- [A survey on deep learning in medical image registration: new technologies, uncertainty, evaluation metrics, and beyond](https://mlsanity.com/paper/2307.15615/) (2023)
- [Unsupervised Deformable Registration for Multi-Modal Images via Disentangled Representations](https://mlsanity.com/paper/1903.09331/) (2019)
- [Networks for Joint Affine and Non-parametric Image Registration](https://mlsanity.com/paper/1903.08811/) (2019)
- [textttGradICON: Approximate Diffeomorphisms via Gradient Inverse Consistency](https://mlsanity.com/paper/2206.05897/) (2022)
- [The Brain Tumor Sequence Registration (BraTS-Reg) Challenge: Establishing Correspondence Between Pre-Operative and Follow-up MRI Scans of Diffuse Glioma Patients](https://mlsanity.com/paper/2112.06979/) (2021)
- [ICON: Learning Regular Maps Through Inverse Consistency](https://mlsanity.com/paper/2105.04459/) (2021)
- [Adversarial Uni- and Multi-modal Stream Networks for Multimodal Image Registration](https://mlsanity.com/paper/2007.02790/) (2020)
- [A Deep Metric for Multimodal Registration](https://mlsanity.com/paper/1609.05396/) (2016)
- [Modality-Agnostic Structural Image Representation Learning for Deformable Multi-Modality Medical Image Registration](https://mlsanity.com/paper/2402.18933/) (2024)
- [Deep Learning based Inter-Modality Image Registration Supervised by Intra-Modality Similarity](https://mlsanity.com/paper/1804.10735/) (2018)
- [Adversarial Image Registration with Application for MR and TRUS Image Fusion](https://mlsanity.com/paper/1804.11024/) (2018)
- [SAMConvex: Fast Discrete Optimization for CT Registration using Self-supervised Anatomical Embedding and Correlation Pyramid](https://mlsanity.com/paper/2307.09727/) (2023)
- [SAME++: A Self-supervised Anatomical eMbeddings Enhanced medical image registration framework using stable sampling and regularized transformation](https://mlsanity.com/paper/2311.14986/) (2023)
- [Fast Predictive Multimodal Image Registration](https://mlsanity.com/paper/1703.10902/) (2017)

## Built on by (most cited first, 16 of 16)

- [Improving Generalization of Medical Image Registration Foundation Model](https://mlsanity.com/paper/2505.06527/) (2025)
- [Zero-shot Domain Generalization of Foundational Models for 3D Medical Image Segmentation: An Experimental Study](https://mlsanity.com/paper/2503.22862/) (2025)
- [CABLD: Contrast-Agnostic Brain Landmark Detection with Consistency-Based Regularization](https://mlsanity.com/paper/2411.17845/) (2024)
- [On The Robustness of Foundational 3D Medical Image Segmentation Models Against Imprecise Visual Prompts](https://mlsanity.com/paper/2601.16383/) (2026)
- [Implicit Deformable Medical Image Registration with Learnable Kernels](https://mlsanity.com/paper/2506.02150/) (2025)
- [Mono-Modalizing Extremely Heterogeneous Multi-Modal Medical Image Registration](https://mlsanity.com/paper/2506.15596/) (2025)
- [MedSapiens: Taking a Pose to Rethink Medical Imaging Landmark Detection](https://mlsanity.com/paper/2511.04255/) (2025)
- [A Steerable Deep Network for Model-Free Diffusion MRI Registration](https://mlsanity.com/paper/2501.04794/) (2025)
- [Downstream Analysis of Foundational Medical Vision Models for Disease Progression](https://mlsanity.com/paper/2503.16842/) (2025)
- [OncoReg: Medical Image Registration for Oncological Challenges](https://mlsanity.com/paper/2503.23179/) (2025)
- [Test Time Optimized Generalized AI-based Medical Image Registration Method](https://mlsanity.com/paper/2512.14556/) (2025)
- [Data-driven registration and modeling of brain deformation for image-guided neurosurgery](https://mlsanity.com/paper/2602.10155/) (2026)
- [Foundation Models for Medical Imaging: Status, Challenges, and Directions](https://mlsanity.com/paper/2602.15913/) (2026)
- [Adapting Frozen Mono-modal Backbones for Multi-modal Registration via Contrast-Agnostic Instance Optimization](https://mlsanity.com/paper/2603.26393/) (2026)
- [XRF-to-Optical Field-of-View Localization with Vision Language Models](https://mlsanity.com/paper/2608.18309/) (2026)
- [Benchmarking Intra-Patient 3D Deformable Multimodal Image Registration](https://mlsanity.com/paper/2609.15669/) (2026)

## Abstract (arXiv)

Modern medical image registration approaches predict deformations using deep networks. These approaches achieve state-of-the-art (SOTA) registration accuracy and are generally fast. However, deep learning (DL) approaches are, in contrast to conventional non-deep-learning-based approaches, anatomy-specific. Recently, a universal deep registration approach, uniGradICON, has been proposed. However, uniGradICON focuses on monomodal image registration. In this work, we therefore develop multiGradICON as a first step towards universal *multimodal* medical image registration. Specifically, we show that 1) we can train a DL registration model that is suitable for monomodal *and* multimodal registration; 2) loss function randomization can increase multimodal registration accuracy; and 3) training a model with multimodal data helps multimodal generalization. Our code and the multiGradICON model are available at https://github.com/uncbiag/uniGradICON.