# Deep Learning based Inter-Modality Image Registration Supervised by Intra-Modality Similarity

arXiv: https://arxiv.org/abs/1804.10735 · HTML: https://mlsanity.com/paper/1804.10735/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Apr 2018.

## Built on by (most cited first, 16 of 16)

- [Unsupervised Misaligned Infrared and Visible Image Fusion via Cross-Modality Image Generation and Registration](https://mlsanity.com/paper/2205.11876/) (2022)
- [DiffuseMorph: Unsupervised Deformable Image Registration Using Diffusion Model](https://mlsanity.com/paper/2112.05149/) (2021)
- [Infrared and Visible Image Fusion: From Data Compatibility to Task Adaption](https://mlsanity.com/paper/2501.10761/) (2025)
- [Improving Misaligned Multi-modality Image Fusion with One-stage Progressive Dense Registration](https://mlsanity.com/paper/2308.11165/) (2023)
- [Data-Efficient Unsupervised Interpolation Without Any Intermediate Frame for 4D Medical Images](https://mlsanity.com/paper/2404.01464/) (2024)
- [multiGradICON: A Foundation Model for Multimodal Medical Image Registration](https://mlsanity.com/paper/2408.00221/) (2024)
- [Joint Registration and Segmentation via Multi-Task Learning for Adaptive Radiotherapy of Prostate Cancer](https://mlsanity.com/paper/2105.01844/) (2021)
- [Deep Learning for Medical Image Registration: A Comprehensive Review](https://mlsanity.com/paper/2204.11341/) (2022)
- [A Transformer-based Network for Deformable Medical Image Registration](https://mlsanity.com/paper/2202.12104/) (2022)
- [Unsupervised Image Registration Towards Enhancing Performance and Explainability in Cardiac And Brain Image Analysis](https://mlsanity.com/paper/2203.03638/) (2022)
- [DPCN++: Differentiable Phase Correlation Network for Versatile Pose Registration](https://mlsanity.com/paper/2206.05707/) (2022)
- [Automatic laminectomy cutting plane planning based on artificial intelligence in robot assisted laminectomy surgery](https://mlsanity.com/paper/2312.17266/) (2023)
- [Subcortical Masks Generation in CT Images via Ensemble-Based Cross-Domain Label Transfer](https://mlsanity.com/paper/2508.11450/) (2025)
- [Surrogate Supervision for Robust and Generalizable Deformable Image Registration](https://mlsanity.com/paper/2509.09869/) (2025)
- [MorphSeek: Fine-grained Latent Representation-Level Policy Optimization for Deformable Image Registration](https://mlsanity.com/paper/2511.17392/) (2025)
- [Domain-Shift Immunity in Deep Deformable Registration via Local Feature Representations](https://mlsanity.com/paper/2512.23142/) (2025)

## Abstract (arXiv)

Non-rigid inter-modality registration can facilitate accurate information fusion from different modalities, but it is challenging due to the very different image appearances across modalities. In this paper, we propose to train a non-rigid inter-modality image registration network, which can directly predict the transformation field from the input multimodal images, such as CT and MR images. In particular, the training of our inter-modality registration network is supervised by intra-modality similarity metric based on the available paired data, which is derived from a pre-aligned CT and MR dataset. Specifically, in the training stage, to register the input CT and MR images, their similarity is evaluated on the warped MR image and the MR image that is paired with the input CT. So that, the intra-modality similarity metric can be directly applied to measure whether the input CT and MR images are well registered. Moreover, we use the idea of dual-modality fashion, in which we measure the similarity on both CT modality and MR modality. In this way, the complementary anatomies in both modalities can be jointly considered to more accurately train the inter-modality registration network. In the testing stage, the trained inter-modality registration network can be directly applied to register the new multimodal images without any paired data. Experimental results have shown that, the proposed method can achieve promising accuracy and efficiency for the challenging non-rigid inter-modality registration task and also outperforms the state-of-the-art approaches.