# Is Image-to-Image Translation the Panacea for Multimodal Image Registration? A Comparative Study

arXiv: https://arxiv.org/abs/2103.16262 · HTML: https://mlsanity.com/paper/2103.16262/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 96 references that are in the map.
- First posted on arXiv 30 Mar 2021.

## Builds on (most cited first, 30 of 37)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Learning deep representations by mutual information estimation and maximization](https://mlsanity.com/paper/1808.06670/) (2018)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [StarGAN v2: Diverse Image Synthesis for Multiple Domains](https://mlsanity.com/paper/1912.01865/) (2019)
- [Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks](https://mlsanity.com/paper/1703.10593/) (2017)
- [VoxelMorph: A Learning Framework for Deformable Medical Image Registration](https://mlsanity.com/paper/1809.05231/) (2018)
- [Unsupervised Image-to-Image Translation Networks](https://mlsanity.com/paper/1703.00848/) (2017)
- [DualGAN: Unsupervised Dual Learning for Image-to-Image Translation](https://mlsanity.com/paper/1704.02510/) (2017)
- [Discriminative Unsupervised Feature Learning with Exemplar Convolutional Neural Networks](https://mlsanity.com/paper/1406.6909/) (2014)
- [Diverse Image-to-Image Translation via Disentangled Representations](https://mlsanity.com/paper/1808.00948/) (2018)
- [Learning to Discover Cross-Domain Relations with Generative Adversarial Networks](https://mlsanity.com/paper/1703.05192/) (2017)
- [DRIT++: Diverse Image-to-Image Translation via Disentangled Representations](https://mlsanity.com/paper/1905.01270/) (2019)
- [A Deep Learning Framework for Unsupervised Affine and Deformable Image Registration](https://mlsanity.com/paper/1809.06130/) (2018)
- [Mode Seeking Generative Adversarial Networks for Diverse Image Synthesis](https://mlsanity.com/paper/1903.05628/) (2019)
- [Unsupervised Cross-Domain Image Generation](https://mlsanity.com/paper/1611.02200/) (2016)
- [Deep Learning in Medical Image Registration: A Survey](https://mlsanity.com/paper/1903.02026/) (2019)
- [Unsupervised Learning for Fast Probabilistic Diffeomorphic Registration](https://mlsanity.com/paper/1805.04605/) (2018)
- [Cross-domain Correspondence Learning for Exemplar-based Image Translation](https://mlsanity.com/paper/2004.05571/) (2020)
- [Image-to-Image Translation: Methods and Applications](https://mlsanity.com/paper/2101.08629/) (2021)
- [Quicksilver: Fast Predictive Image Registration - a Deep Learning Approach](https://mlsanity.com/paper/1703.10908/) (2017)
- [Deep Learning in Medical Image Registration: A Review](https://mlsanity.com/paper/1912.12318/) (2019)
- [Deep learning-based virtual histology staining using auto-fluorescence of label-free tissue](https://mlsanity.com/paper/1803.11293/) (2018)
- [Unsupervised Multi-Modal Image Registration via Geometry Preserving Image-to-Image Translation](https://mlsanity.com/paper/2003.08073/) (2020)
- [Unsupervised Deformable Registration for Multi-Modal Images via Disentangled Representations](https://mlsanity.com/paper/1903.09331/) (2019)
- [CoMIR: Contrastive Multimodal Image Representation for Registration](https://mlsanity.com/paper/2006.06325/) (2020)
- [Unsupervised Medical Image Translation Using Cycle-MedGAN](https://mlsanity.com/paper/1903.03374/) (2019)
- [Learning Fixed Points in Generative Adversarial Networks: From Image-to-Image Translation to Disease Detection and Localization](https://mlsanity.com/paper/1908.06965/) (2019)
- [Vox2Vox: 3D-GAN for Brain Tumour Segmentation](https://mlsanity.com/paper/2003.13653/) (2020)

## Built on by (most cited first, 9 of 9)

- [Similarity and Quality Metrics for MR Image-To-Image Translation](https://mlsanity.com/paper/2405.08431/) (2024)
- [ContraReg: Contrastive Learning of Multi-modality Unsupervised Deformable Image Registration](https://mlsanity.com/paper/2206.13434/) (2022)
- [Fast computation of mutual information in the frequency domain with applications to global multimodal image alignment](https://mlsanity.com/paper/2106.14699/) (2021)
- [Deformation equivariant cross-modality image synthesis with paired non-aligned training data](https://mlsanity.com/paper/2208.12491/) (2022)
- [Infusing Lattice Symmetry Priors in Attention Mechanisms for Sample-Efficient Abstract Geometric Reasoning](https://mlsanity.com/paper/2306.03175/) (2023)
- [Can representation learning for multimodal image registration be improved by supervision of intermediate layers?](https://mlsanity.com/paper/2303.00403/) (2023)
- [Multi-modal deformable image registration using untrained neural networks](https://mlsanity.com/paper/2411.02672/) (2024)
- [Multimodal Diffeomorphic Registration with Neural ODEs and Structural Descriptors](https://mlsanity.com/paper/2512.22689/) (2025)
- [Benchmarking Intra-Patient 3D Deformable Multimodal Image Registration](https://mlsanity.com/paper/2609.15669/) (2026)

## Abstract (arXiv)

Despite current advancement in the field of biomedical image processing, propelled by the deep learning revolution, multimodal image registration, due to its several challenges, is still often performed manually by specialists. The recent success of image-to-image (I2I) translation in computer vision applications and its growing use in biomedical areas provide a tempting possibility of transforming the multimodal registration problem into a, potentially easier, monomodal one. We conduct an empirical study of the applicability of modern I2I translation methods for the task of rigid registration of multimodal biomedical and medical 2D and 3D images. We compare the performance of four Generative Adversarial Network (GAN)-based I2I translation methods and one contrastive representation learning method, subsequently combined with two representative monomodal registration methods, to judge the effectiveness of modality translation for multimodal image registration. We evaluate these method combinations on four publicly available multimodal (2D and 3D) datasets and compare them with the performance of registration achieved by several well-known approaches acting directly on multimodal image data. Our results suggest that, although I2I translation may be helpful when the modalities to register are clearly correlated, registration of modalities which express distinctly different properties of the sample is not well handled by the I2I translation approach. The evaluated representation learning method, which aims to find abstract image-like representations of the information shared between the modalities, manages better, and so does the Mutual Information maximisation approach, acting directly on the original multimodal images. We share our complete experimental setup as open-source (https://github.com/MIDA-group/MultiRegEval).