# Learning Deep Similarity Metric for 3D MR-TRUS Registration

arXiv: https://arxiv.org/abs/1806.04548 · HTML: https://mlsanity.com/paper/1806.04548/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Jun 2018.

## Built on by (most cited first, 24 of 24)

- [Recent advances and clinical applications of deep learning in medical image analysis](https://mlsanity.com/paper/2105.13381/) (2021)
- [A survey on deep learning in medical image registration: new technologies, uncertainty, evaluation metrics, and beyond](https://mlsanity.com/paper/2307.15615/) (2023)
- [Cross-modal Attention for MRI and Ultrasound Volume Registration](https://mlsanity.com/paper/2107.04548/) (2021)
- [End-to-end Ultrasound Frame to Volume Registration](https://mlsanity.com/paper/2107.06449/) (2021)
- [General Purpose Image Encoder DINOv2 for Medical Image Registration](https://mlsanity.com/paper/2402.15687/) (2024)
- [Real-time multimodal image registration with partial intraoperative point-set data](https://mlsanity.com/paper/2109.05023/) (2021)
- [DISA: DIfferentiable Similarity Approximation for Universal Multimodal Registration](https://mlsanity.com/paper/2307.09931/) (2023)
- [Semantic similarity metrics for learned image registration](https://mlsanity.com/paper/2104.10051/) (2021)
- [Deep Learning for Medical Image Registration: A Comprehensive Review](https://mlsanity.com/paper/2204.11341/) (2022)
- [MAD: Modality Agnostic Distance Measure for Image Registration](https://mlsanity.com/paper/2309.02875/) (2023)
- [IMPACT: A Generic Semantic Loss for Multimodal Medical Image Registration](https://mlsanity.com/paper/2503.24121/) (2025)
- [Learning Multi-Modal Volumetric Prostate Registration with Weak Inter-Subject Spatial Correspondence](https://mlsanity.com/paper/2102.04938/) (2021)
- [TRUSTED: The Paired 3D Transabdominal Ultrasound and CT Human Data for Kidney Segmentation and Registration Research](https://mlsanity.com/paper/2310.12646/) (2023)
- [A Narrative Review of Image Processing Techniques Related to Prostate Ultrasound](https://mlsanity.com/paper/2407.00678/) (2024)
- [Domain adaptation strategies for 3D reconstruction of the lumbar spine using real fluoroscopy data](https://mlsanity.com/paper/2401.16027/) (2024)
- [Distance Map Supervised Landmark Localization for MR-TRUS Registration](https://mlsanity.com/paper/2210.05738/) (2022)
- [General Vision Encoder Features as Guidance in Medical Image Registration](https://mlsanity.com/paper/2407.13311/) (2024)
- [D-Net: Siamese based Network with Mutual Attention for Volume Alignment](https://mlsanity.com/paper/2101.10248/) (2021)
- [The use of deep learning in interventional radiotherapy (brachytherapy): a review with a focus on open source and open data](https://mlsanity.com/paper/2205.07516/) (2022)
- [DPCN++: Differentiable Phase Correlation Network for Versatile Pose Registration](https://mlsanity.com/paper/2206.05707/) (2022)
- [Deformable Image Registration using Unsupervised Deep Learning for CBCT-guided Abdominal Radiotherapy](https://mlsanity.com/paper/2208.13686/) (2022)
- [Salient Region Matching for Fully Automated MR-TRUS Registration](https://mlsanity.com/paper/2501.03510/) (2025)
- [MorphSeek: Fine-grained Latent Representation-Level Policy Optimization for Deformable Image Registration](https://mlsanity.com/paper/2511.17392/) (2025)
- [Benchmarking Intra-Patient 3D Deformable Multimodal Image Registration](https://mlsanity.com/paper/2609.15669/) (2026)

## Abstract (arXiv)

Purpose: The fusion of transrectal ultrasound (TRUS) and magnetic resonance (MR) images for guiding targeted prostate biopsy has significantly improved the biopsy yield of aggressive cancers. A key component of MR-TRUS fusion is image registration. However, it is very challenging to obtain a robust automatic MR-TRUS registration due to the large appearance difference between the two imaging modalities. The work presented in this paper aims to tackle this problem by addressing two challenges: (i) the definition of a suitable similarity metric and (ii) the determination of a suitable optimization strategy. Methods: This work proposes the use of a deep convolutional neural network to learn a similarity metric for MR-TRUS registration. We also use a composite optimization strategy that explores the solution space in order to search for a suitable initialization for the second-order optimization of the learned metric. Further, a multi-pass approach is used in order to smooth the metric for optimization. Results: The learned similarity metric outperforms the classical mutual information and also the state-of-the-art MIND feature based methods. The results indicate that the overall registration framework has a large capture range. The proposed deep similarity metric based approach obtained a mean TRE of 3.86mm (with an initial TRE of 16mm) for this challenging problem. Conclusion: A similarity metric that is learned using a deep neural network can be used to assess the quality of any given image registration and can be used in conjunction with the aforementioned optimization framework to perform automatic registration that is robust to poor initialization.