# Unsupervised Deformable Registration for Multi-Modal Images via Disentangled Representations

arXiv: https://arxiv.org/abs/1903.09331 · HTML: https://mlsanity.com/paper/1903.09331/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Mar 2019.

## Built on by (most cited first, 30 of 52)

- [Breaking the Dilemma of Medical Image-to-image Translation](https://mlsanity.com/paper/2110.06465/) (2021)
- [A survey on deep learning in medical image registration: new technologies, uncertainty, evaluation metrics, and beyond](https://mlsanity.com/paper/2307.15615/) (2023)
- [Learning Disentangled Representations in the Imaging Domain](https://mlsanity.com/paper/2108.12043/) (2021)
- [A Robust and Interpretable Deep Learning Framework for Multi-modal Registration via Keypoints](https://mlsanity.com/paper/2304.09941/) (2023)
- [Representation Disentanglement for Multi-modal brain MR Analysis](https://mlsanity.com/paper/2102.11456/) (2021)
- [Generative Myocardial Motion Tracking via Latent Space Exploration with Biomechanics-informed Prior](https://mlsanity.com/paper/2206.03830/) (2022)
- [FedDis: Disentangled Federated Learning for Unsupervised Brain Pathology Segmentation](https://mlsanity.com/paper/2103.03705/) (2021)
- [Multi-Modality Cardiac Image Computing: A Survey](https://mlsanity.com/paper/2208.12881/) (2022)
- [Anatomy-aware and acquisition-agnostic joint registration with SynthMorph](https://mlsanity.com/paper/2301.11329/) (2023)
- [Multi-scale Neural ODEs for 3D Medical Image Registration](https://mlsanity.com/paper/2106.08493/) (2021)
- [multiGradICON: A Foundation Model for Multimodal Medical Image Registration](https://mlsanity.com/paper/2408.00221/) (2024)
- [Unsupervised Multi-Modal Medical Image Registration via Discriminator-Free Image-to-Image Translation](https://mlsanity.com/paper/2204.13656/) (2022)
- [Is Image-to-Image Translation the Panacea for Multimodal Image Registration? A Comparative Study](https://mlsanity.com/paper/2103.16262/) (2021)
- [ContraReg: Contrastive Learning of Multi-modality Unsupervised Deformable Image Registration](https://mlsanity.com/paper/2206.13434/) (2022)
- [Cross-Modality Image Registration using a Training-Time Privileged Third Modality](https://mlsanity.com/paper/2207.12901/) (2022)
- [Cross-Modal Vertical Federated Learning for MRI Reconstruction](https://mlsanity.com/paper/2306.02673/) (2023)
- [Anatomy-guided Multimodal Registration by Learning Segmentation without Ground Truth: Application to Intraprocedural CBCT/MR Liver Segmentation and Registration](https://mlsanity.com/paper/2104.07056/) (2021)
- [Cross-Modality Multi-Atlas Segmentation via Deep Registration and Label Fusion](https://mlsanity.com/paper/2202.02000/) (2022)
- [Deep Learning for Medical Image Registration: A Comprehensive Review](https://mlsanity.com/paper/2204.11341/) (2022)
- [GLOWin: A Flow-based Invertible Generative Framework for Learning Disentangled Feature Representations in Medical Images](https://mlsanity.com/paper/2103.10868/) (2021)
- [Going Beyond Saliency Maps: Training Deep Models to Interpret Deep Models](https://mlsanity.com/paper/2102.08239/) (2021)
- [Indescribable Multi-modal Spatial Evaluator](https://mlsanity.com/paper/2303.00369/) (2023)
- [Conditional Deformable Image Registration with Spatially-Variant and Adaptive Regularization](https://mlsanity.com/paper/2303.10700/) (2023)
- [Robust joint registration of multiple stains and MRI for multimodal 3D histology reconstruction: Application to the Allen human brain atlas](https://mlsanity.com/paper/2104.14873/) (2021)
- [Synth-by-Reg (SbR): Contrastive learning for synthesis-based registration of paired images](https://mlsanity.com/paper/2107.14449/) (2021)
- [MICS : Multi-steps, Inverse Consistency and Symmetric deep learning registration network](https://mlsanity.com/paper/2111.12123/) (2021)
- [MetaMorph: Learning Metamorphic Image Transformation With Appearance Changes](https://mlsanity.com/paper/2303.04849/) (2023)
- [Breaking Modality Disparity: Harmonized Representation for Infrared and Visible Image Registration](https://mlsanity.com/paper/2304.05646/) (2023)
- [BInGo: Bayesian Intrinsic Groupwise Registration via Explicit Hierarchical Disentanglement](https://mlsanity.com/paper/2206.02377/) (2022)
- [Medical Image Registration and Its Application in Retinal Images: A Review](https://mlsanity.com/paper/2403.16502/) (2024)

## Abstract (arXiv)

We propose a fully unsupervised multi-modal deformable image registration method (UMDIR), which does not require any ground truth deformation fields or any aligned multi-modal image pairs during training. Multi-modal registration is a key problem in many medical image analysis applications. It is very challenging due to complicated and unknown relationships between different modalities. In this paper, we propose an unsupervised learning approach to reduce the multi-modal registration problem to a mono-modal one through image disentangling. In particular, we decompose images of both modalities into a common latent shape space and separate latent appearance spaces via an unsupervised multi-modal image-to-image translation approach. The proposed registration approach is then built on the factorized latent shape code, with the assumption that the intrinsic shape deformation existing in original image domain is preserved in this latent space. Specifically, two metrics have been proposed for training the proposed network: a latent similarity metric defined in the common shape space and a learningbased image similarity metric based on an adversarial loss. We examined different variations of our proposed approach and compared them with conventional state-of-the-art multi-modal registration methods. Results show that our proposed methods achieve competitive performance against other methods at substantially reduced computation time.