# Semantic similarity metrics for learned image registration

arXiv: https://arxiv.org/abs/2104.10051 · HTML: https://mlsanity.com/paper/2104.10051/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 39 references that are in the map.
- First posted on arXiv 20 Apr 2021.

## Builds on (most cited first, 21 of 21)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets](https://mlsanity.com/paper/1606.03657/) (2016)
- [VoxelMorph: A Learning Framework for Deformable Medical Image Registration](https://mlsanity.com/paper/1809.05231/) (2018)
- [A Deep Learning Framework for Unsupervised Affine and Deformable Image Registration](https://mlsanity.com/paper/1809.06130/) (2018)
- [Unsupervised Learning for Fast Probabilistic Diffeomorphic Registration](https://mlsanity.com/paper/1805.04605/) (2018)
- [Quicksilver: Fast Predictive Image Registration - a Deep Learning Approach](https://mlsanity.com/paper/1703.10908/) (2017)
- [DeepAtlas: Joint Semi-Supervised Learning of Image Registration and Segmentation](https://mlsanity.com/paper/1904.08465/) (2019)
- [Learning Conditional Deformable Templates with Convolutional Networks](https://mlsanity.com/paper/1908.02738/) (2019)
- [Dual-Stream Pyramid Registration Network](https://mlsanity.com/paper/1909.11966/) (2019)
- [CoMIR: Contrastive Multimodal Image Representation for Registration](https://mlsanity.com/paper/2006.06325/) (2020)
- [A Loss Function for Generative Neural Networks Based on Watson's Perceptual Model](https://mlsanity.com/paper/2006.15057/) (2020)
- [Image-and-Spatial Transformer Networks for Structure-Guided Image Registration](https://mlsanity.com/paper/1907.09200/) (2019)
- [Non-Rigid Image Registration Using Self-Supervised Fully Convolutional Networks without Training Data](https://mlsanity.com/paper/1801.04012/) (2018)
- [Learning Deep Similarity Metric for 3D MR-TRUS Registration](https://mlsanity.com/paper/1806.04548/) (2018)
- [A Deep Metric for Multimodal Registration](https://mlsanity.com/paper/1609.05396/) (2016)
- [Probabilistic Multilayer Regularization Network for Unsupervised 3D Brain Image Registration](https://mlsanity.com/paper/1907.01922/) (2019)
- [Conditional Segmentation in Lieu of Image Registration](https://mlsanity.com/paper/1907.00438/) (2019)

## Built on by (most cited first, 9 of 9)

- [A survey on deep learning in medical image registration: new technologies, uncertainty, evaluation metrics, and beyond](https://mlsanity.com/paper/2307.15615/) (2023)
- [Neuralizer: General Neuroimage Analysis without Re-Training](https://mlsanity.com/paper/2305.02644/) (2023)
- [ContraReg: Contrastive Learning of Multi-modality Unsupervised Deformable Image Registration](https://mlsanity.com/paper/2206.13434/) (2022)
- [Guiding Registration with Emergent Similarity from Pre-Trained Diffusion Models](https://mlsanity.com/paper/2506.02419/) (2025)
- [Medical Image Registration and Its Application in Retinal Images: A Review](https://mlsanity.com/paper/2403.16502/) (2024)
- [Is Registering Raw Tagged-MR Enough for Strain Estimation in the Era of Deep Learning?](https://mlsanity.com/paper/2401.17571/) (2024)
- [Spot the Difference: Detection of Topological Changes via Geometric Alignment](https://mlsanity.com/paper/2106.08233/) (2021)
- [Subspace-Guided Semantic and Topological Invariant Registration for Annotation-Free Ultrasound Plane Quality Control](https://mlsanity.com/paper/2605.25396/) (2026)
- [Benchmarking Intra-Patient 3D Deformable Multimodal Image Registration](https://mlsanity.com/paper/2609.15669/) (2026)

## Abstract (arXiv)

We propose a semantic similarity metric for image registration. Existing metrics like Euclidean Distance or Normalized Cross-Correlation focus on aligning intensity values, giving difficulties with low intensity contrast or noise. Our approach learns dataset-specific features that drive the optimization of a learning-based registration model. We train both an unsupervised approach using an auto-encoder, and a semi-supervised approach using supplemental segmentation data to extract semantic features for image registration. Comparing to existing methods across multiple image modalities and applications, we achieve consistently high registration accuracy. A learned invariance to noise gives smoother transformations on low-quality images.