# Mutual information neural estimation for unsupervised multi-modal registration of brain images

arXiv: https://arxiv.org/abs/2201.10305 · HTML: https://mlsanity.com/paper/2201.10305/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 15 references that are in the map.
- First posted on arXiv 25 Jan 2022.

## Builds on (most cited first, 5 of 5)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [MINE: Mutual Information Neural Estimation](https://mlsanity.com/paper/1801.04062/) (2018)
- [VoxelMorph: A Learning Framework for Deformable Medical Image Registration](https://mlsanity.com/paper/1809.05231/) (2018)
- [Unsupervised Learning of Probabilistic Diffeomorphic Registration for Images and Surfaces](https://mlsanity.com/paper/1903.03545/) (2019)
- [DRMIME: Differentiable Mutual Information and Matrix Exponential for Multi-Resolution Image Registration](https://mlsanity.com/paper/2001.09865/) (2020)

## Built on by (most cited first, 3 of 3)

- [Unsupervised Domain Adaptation for Cardiac Segmentation: Towards Structure Mutual Information Maximization](https://mlsanity.com/paper/2204.09334/) (2022)
- [Indescribable Multi-modal Spatial Evaluator](https://mlsanity.com/paper/2303.00369/) (2023)
- [Structure-Guided MR-to-CT Synthesis with Spatial and Semantic Alignments for Attenuation Correction of Whole-Body PET/MR Imaging](https://mlsanity.com/paper/2411.17488/) (2024)

## Abstract (arXiv)

Many applications in image-guided surgery and therapy require fast and reliable non-linear, multi-modal image registration. Recently proposed unsupervised deep learning-based registration methods have demonstrated superior performance compared to iterative methods in just a fraction of the time. Most of the learning-based methods have focused on mono-modal image registration. The extension to multi-modal registration depends on the use of an appropriate similarity function, such as the mutual information (MI). We propose guiding the training of a deep learning-based registration method with MI estimation between an image-pair in an end-to-end trainable network. Our results show that a small, 2-layer network produces competitive results in both mono- and multi-modal registration, with sub-second run-times. Comparisons to both iterative and deep learning-based methods show that our MI-based method produces topologically and qualitatively superior results with an extremely low rate of non-diffeomorphic transformations. Real-time clinical application will benefit from a better visual matching of anatomical structures and less registration failures/outliers.