# Benchmarking Intra-Patient 3D Deformable Multimodal Image Registration

arXiv: https://arxiv.org/abs/2609.15669 · HTML: https://mlsanity.com/paper/2609.15669/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 33 of its 75 references that are in the map.
- First posted on arXiv 14 Sep 2026.

## Builds on (most cited first, 30 of 33)

- [VoxelMorph: A Learning Framework for Deformable Medical Image Registration](https://mlsanity.com/paper/1809.05231/) (2018)
- [Unsupervised Learning of Probabilistic Diffeomorphic Registration for Images and Surfaces](https://mlsanity.com/paper/1903.03545/) (2019)
- [A Deep Learning Framework for Unsupervised Affine and Deformable Image Registration](https://mlsanity.com/paper/1809.06130/) (2018)
- [Learn2Reg: comprehensive multi-task medical image registration challenge, dataset and evaluation in the era of deep learning](https://mlsanity.com/paper/2112.04489/) (2021)
- [Deep Learning in Medical Image Registration: A Survey](https://mlsanity.com/paper/1903.02026/) (2019)
- [Deep Learning in Medical Image Registration: A Review](https://mlsanity.com/paper/1912.12318/) (2019)
- [Weakly-Supervised Convolutional Neural Networks for Multimodal Image Registration](https://mlsanity.com/paper/1807.03361/) (2018)
- [ViT-V-Net: Vision Transformer for Unsupervised Volumetric Medical Image Registration](https://mlsanity.com/paper/2104.06468/) (2021)
- [Breaking the Dilemma of Medical Image-to-image Translation](https://mlsanity.com/paper/2110.06465/) (2021)
- [CycleMorph: Cycle Consistent Unsupervised Deformable Image Registration](https://mlsanity.com/paper/2008.05772/) (2020)
- [A survey on deep learning in medical image registration: new technologies, uncertainty, evaluation metrics, and beyond](https://mlsanity.com/paper/2307.15615/) (2023)
- [CoMIR: Contrastive Multimodal Image Representation for Registration](https://mlsanity.com/paper/2006.06325/) (2020)
- [textttGradICON: Approximate Diffeomorphisms via Gradient Inverse Consistency](https://mlsanity.com/paper/2206.05897/) (2022)
- [MatchAnything: Universal Cross-Modality Image Matching with Large-Scale Pre-Training](https://mlsanity.com/paper/2501.07556/) (2025)
- [Metric Learning for Image Registration](https://mlsanity.com/paper/1904.09524/) (2019)
- [Adversarial Uni- and Multi-modal Stream Networks for Multimodal Image Registration](https://mlsanity.com/paper/2007.02790/) (2020)
- [Learning Deep Similarity Metric for 3D MR-TRUS Registration](https://mlsanity.com/paper/1806.04548/) (2018)
- [SAME: Deformable Image Registration based on Self-supervised Anatomical Embeddings](https://mlsanity.com/paper/2109.11572/) (2021)
- [A Deep Metric for Multimodal Registration](https://mlsanity.com/paper/1609.05396/) (2016)
- [Modality-Agnostic Structural Image Representation Learning for Deformable Multi-Modality Medical Image Registration](https://mlsanity.com/paper/2402.18933/) (2024)
- [multiGradICON: A Foundation Model for Multimodal Medical Image Registration](https://mlsanity.com/paper/2408.00221/) (2024)
- [DISA: DIfferentiable Similarity Approximation for Universal Multimodal Registration](https://mlsanity.com/paper/2307.09931/) (2023)
- [Is Image-to-Image Translation the Panacea for Multimodal Image Registration? A Comparative Study](https://mlsanity.com/paper/2103.16262/) (2021)
- [Semantic similarity metrics for learned image registration](https://mlsanity.com/paper/2104.10051/) (2021)
- [Synthesis and Inpainting-Based MR-CT Registration for Image-Guided Thermal Ablation of Liver Tumors](https://mlsanity.com/paper/1907.13020/) (2019)
- [Metamorphic image registration using a semi-Lagrangian scheme](https://mlsanity.com/paper/2106.08817/) (2021)
- [A deep residual learning implementation of Metamorphosis](https://mlsanity.com/paper/2202.00676/) (2022)
- [Weakly-supervised Biomechanically-constrained CT/MRI Registration of the Spine](https://mlsanity.com/paper/2205.07568/) (2022)
- [Guiding Registration with Emergent Similarity from Pre-Trained Diffusion Models](https://mlsanity.com/paper/2506.02419/) (2025)
- [Learn2Reg 2024: New Benchmark Datasets Driving Progress on New Challenges](https://mlsanity.com/paper/2509.01217/) (2025)

## Abstract (arXiv)

Multimodal image registration is a key component of many clinical workflows, yet it remains challenging because corresponding anatomical structures often exhibit substantially different image intensities across modalities. In this work, we present a comprehensive benchmark of intra-patient 3D multimodal deformable registration methods across three datasets covering different anatomical regions and difficulty levels, including both synthetic deformation recovery and real clinical scenarios. We evaluate classical optimization-based approaches and modern learning-based methods, including recent deep learning and foundation models, using complementary metrics: Average Dice similarity coefficient (DSC), average 95th-percentile Hausdorff distance (HD95), and a modality-independent structural similarity measure based on the MIND self-similarity context (MIND-SSC). Results show high variability across datasets, with learning-based methods demonstrating superior performance on large synthetic benchmarks, while only limited improvements are observed in real pelvic registration. A key finding of this study is the consistent disagreement between geometric metrics (DSC, HD95) and image-based similarity metrics (MIND-SSC), highlighting that improved overlap does not necessarily imply better global multimodal correspondence. Furthermore, anatomy-guided approaches achieve the highest overlap scores but exhibit degraded performance outside of segmented regions, revealing a trade-off between label-driven alignment and global structural coherence. Overall, our results indicate that no current method achieves robust performance across anatomies and modalities. We demonstrate that intra-patient 3D multimodal registration requires multi-criteria evaluation, including deformation-based metrics, and remains an open problem.