# Unsupervised Multi-Modal Image Registration via Geometry Preserving Image-to-Image Translation

arXiv: https://arxiv.org/abs/2003.08073 · HTML: https://mlsanity.com/paper/2003.08073/

## Key facts

- Cited by 66 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Mar 2020.

## Built on by (most cited first, 30 of 66)

- [Image-to-Image Translation: Methods and Applications](https://mlsanity.com/paper/2101.08629/) (2021)
- [LasHeR: A Large-scale High-diversity Benchmark for RGBT Tracking](https://mlsanity.com/paper/2104.13202/) (2021)
- [Breaking the Dilemma of Medical Image-to-image Translation](https://mlsanity.com/paper/2110.06465/) (2021)
- [Unsupervised Misaligned Infrared and Visible Image Fusion via Cross-Modality Image Generation and Registration](https://mlsanity.com/paper/2205.11876/) (2022)
- [XoFTR: Cross-modal Feature Matching Transformer](https://mlsanity.com/paper/2404.09692/) (2024)
- [Infrared and Visible Image Fusion: From Data Compatibility to Task Adaption](https://mlsanity.com/paper/2501.10761/) (2025)
- [FreeReg: Image-to-Point Cloud Registration Leveraging Pretrained Diffusion Models and Monocular Depth Estimators](https://mlsanity.com/paper/2310.03420/) (2023)
- [Multi-Modality Cardiac Image Computing: A Survey](https://mlsanity.com/paper/2208.12881/) (2022)
- [Multi-scale Neural ODEs for 3D Medical Image Registration](https://mlsanity.com/paper/2106.08493/) (2021)
- [Improving Misaligned Multi-modality Image Fusion with One-stage Progressive Dense Registration](https://mlsanity.com/paper/2308.11165/) (2023)
- [Multi-Modal MRI Reconstruction Assisted with Spatial Alignment Network](https://mlsanity.com/paper/2108.05603/) (2021)
- [Unsupervised Multi-Modal Medical Image Registration via Discriminator-Free Image-to-Image Translation](https://mlsanity.com/paper/2204.13656/) (2022)
- [Learning Mutual Modulation for Self-Supervised Cross-Modal Super-Resolution](https://mlsanity.com/paper/2207.09156/) (2022)
- [Is Image-to-Image Translation the Panacea for Multimodal Image Registration? A Comparative Study](https://mlsanity.com/paper/2103.16262/) (2021)
- [Physics-informed Guided Disentanglement in Generative Networks](https://mlsanity.com/paper/2107.14229/) (2021)
- [ContraReg: Contrastive Learning of Multi-modality Unsupervised Deformable Image Registration](https://mlsanity.com/paper/2206.13434/) (2022)
- [Anatomy-guided Multimodal Registration by Learning Segmentation without Ground Truth: Application to Intraprocedural CBCT/MR Liver Segmentation and Registration](https://mlsanity.com/paper/2104.07056/) (2021)
- [Virtual birefringence imaging and histological staining of amyloid deposits in label-free tissue using autofluorescence microscopy and deep learning](https://mlsanity.com/paper/2403.09100/) (2024)
- [Lesion-Inspired Denoising Network: Connecting Medical Image Denoising and Lesion Detection](https://mlsanity.com/paper/2104.08845/) (2021)
- [Multi-modal Retinal Image Registration Using a Keypoint-Based Vessel Structure Aligning Network](https://mlsanity.com/paper/2207.10506/) (2022)
- [SCPNet: Unsupervised Cross-modal Homography Estimation via Intra-modal Self-supervised Learning](https://mlsanity.com/paper/2407.08148/) (2024)
- [Zero-shot-Learning Cross-Modality Data Translation Through Mutual Information Guided Stochastic Diffusion](https://mlsanity.com/paper/2301.13743/) (2023)
- [SAMScore: A Content Structural Similarity Metric for Image Translation Evaluation](https://mlsanity.com/paper/2305.15367/) (2023)
- [Medical image registration using unsupervised deep neural network: A scoping literature review](https://mlsanity.com/paper/2208.01825/) (2022)
- [Deformation equivariant cross-modality image synthesis with paired non-aligned training data](https://mlsanity.com/paper/2208.12491/) (2022)
- [Indescribable Multi-modal Spatial Evaluator](https://mlsanity.com/paper/2303.00369/) (2023)
- [Synth-by-Reg (SbR): Contrastive learning for synthesis-based registration of paired images](https://mlsanity.com/paper/2107.14449/) (2021)
- [Neural Pre-Processing: A Learning Framework for End-to-end Brain MRI Pre-processing](https://mlsanity.com/paper/2303.12148/) (2023)
- [Breaking Modality Disparity: Harmonized Representation for Infrared and Visible Image Registration](https://mlsanity.com/paper/2304.05646/) (2023)
- [SSHNet: Unsupervised Cross-modal Homography Estimation via Problem Reformulation and Split Optimization](https://mlsanity.com/paper/2409.17993/) (2024)

## Abstract (arXiv)

Many applications, such as autonomous driving, heavily rely on multi-modal data where spatial alignment between the modalities is required. Most multi-modal registration methods struggle computing the spatial correspondence between the images using prevalent cross-modality similarity measures. In this work, we bypass the difficulties of developing cross-modality similarity measures, by training an image-to-image translation network on the two input modalities. This learned translation allows training the registration network using simple and reliable mono-modality metrics. We perform multi-modal registration using two networks - a spatial transformation network and a translation network. We show that by encouraging our translation network to be geometry preserving, we manage to train an accurate spatial transformation network. Compared to state-of-the-art multi-modal methods our presented method is unsupervised, requiring no pairs of aligned modalities for training, and can be adapted to any pair of modalities. We evaluate our method quantitatively and qualitatively on commercial datasets, showing that it performs well on several modalities and achieves accurate alignment.