# Adversarial Uni- and Multi-modal Stream Networks for Multimodal Image Registration

arXiv: https://arxiv.org/abs/2007.02790 · HTML: https://mlsanity.com/paper/2007.02790/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Jul 2020.

## Built on by (most cited first, 25 of 25)

- [A survey on deep learning in medical image registration: new technologies, uncertainty, evaluation metrics, and beyond](https://mlsanity.com/paper/2307.15615/) (2023)
- [ACN: Adversarial Co-training Network for Brain Tumor Segmentation with Missing Modalities](https://mlsanity.com/paper/2106.14591/) (2021)
- [All-Around Real Label Supervision: Cyclic Prototype Consistency Learning for Semi-supervised Medical Image Segmentation](https://mlsanity.com/paper/2109.13930/) (2021)
- [MulViMotion: Shape-aware 3D Myocardial Motion Tracking from Multi-View Cardiac MRI](https://mlsanity.com/paper/2208.00034/) (2022)
- [multiGradICON: A Foundation Model for Multimodal Medical Image Registration](https://mlsanity.com/paper/2408.00221/) (2024)
- [DeepMesh: Mesh-based Cardiac Motion Tracking using Deep Learning](https://mlsanity.com/paper/2309.14306/) (2023)
- [Unsupervised Multi-Modal Medical Image Registration via Discriminator-Free Image-to-Image Translation](https://mlsanity.com/paper/2204.13656/) (2022)
- [Real-time multimodal image registration with partial intraoperative point-set data](https://mlsanity.com/paper/2109.05023/) (2021)
- [Double-Uncertainty Guided Spatial and Temporal Consistency Regularization Weighting for Learning-based Abdominal Registration](https://mlsanity.com/paper/2107.02433/) (2021)
- [Mesh-based 3D Motion Tracking in Cardiac MRI using Deep Learning](https://mlsanity.com/paper/2209.02004/) (2022)
- [Deformer: Towards Displacement Field Learning for Unsupervised Medical Image Registration](https://mlsanity.com/paper/2207.03180/) (2022)
- [Indescribable Multi-modal Spatial Evaluator](https://mlsanity.com/paper/2303.00369/) (2023)
- [Robust joint registration of multiple stains and MRI for multimodal 3D histology reconstruction: Application to the Allen human brain atlas](https://mlsanity.com/paper/2104.14873/) (2021)
- [Synth-by-Reg (SbR): Contrastive learning for synthesis-based registration of paired images](https://mlsanity.com/paper/2107.14449/) (2021)
- [Meta-Learning Initializations for Interactive Medical Image Registration](https://mlsanity.com/paper/2210.15371/) (2022)
- [A Survey on Image-text Multimodal Models](https://mlsanity.com/paper/2309.15857/) (2023)
- [Medical Image Registration and Its Application in Retinal Images: A Review](https://mlsanity.com/paper/2403.16502/) (2024)
- [Leveraging Unsupervised Image Registration for Discovery of Landmark Shape Descriptor](https://mlsanity.com/paper/2111.07009/) (2021)
- [Dual-Branch Squeeze-Fusion-Excitation Module for Cross-Modality Registration of Cardiac SPECT and CT](https://mlsanity.com/paper/2206.05278/) (2022)
- [From Traditional to Deep Learning Approaches in Whole Slide Image Registration: A Methodological Review](https://mlsanity.com/paper/2502.19123/) (2025)
- [Generating Thermal Human Faces for Physiological Assessment Using Thermal Sensor Auxiliary Labels](https://mlsanity.com/paper/2106.08091/) (2021)
- [MambaReg: Mamba-Based Disentangled Convolutional Sparse Coding for Unsupervised Deformable Multi-Modal Image Registration](https://mlsanity.com/paper/2411.01399/) (2024)
- [Towards Better Dermoscopic Image Feature Representation Learning for Melanoma Classification](https://mlsanity.com/paper/2207.07303/) (2022)
- [Data-driven registration and modeling of brain deformation for image-guided neurosurgery](https://mlsanity.com/paper/2602.10155/) (2026)
- [Benchmarking Intra-Patient 3D Deformable Multimodal Image Registration](https://mlsanity.com/paper/2609.15669/) (2026)

## Abstract (arXiv)

Deformable image registration between Computed Tomography (CT) images and Magnetic Resonance (MR) imaging is essential for many image-guided therapies. In this paper, we propose a novel translation-based unsupervised deformable image registration method. Distinct from other translation-based methods that attempt to convert the multimodal problem (e.g., CT-to-MR) into a unimodal problem (e.g., MR-to-MR) via image-to-image translation, our method leverages the deformation fields estimated from both: (i) the translated MR image and (ii) the original CT image in a dual-stream fashion, and automatically learns how to fuse them to achieve better registration performance. The multimodal registration network can be effectively trained by computationally efficient similarity metrics without any ground-truth deformation. Our method has been evaluated on two clinical datasets and demonstrates promising results compared to state-of-the-art traditional and learning-based methods.