# Guiding Registration with Emergent Similarity from Pre-Trained Diffusion Models

arXiv: https://arxiv.org/abs/2506.02419 · HTML: https://mlsanity.com/paper/2506.02419/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models.
- Builds on 35 of its 47 references that are in the map.
- First posted on arXiv 3 Jun 2025.

## Placed in

- Diffusion models / [Medical imaging and segmentation](https://mlsanity.com/diffusion/medical-imaging-and-segmentation/)

## Builds on (most cited first, 30 of 35)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translation](https://mlsanity.com/paper/2211.12572/) (2022)
- [VoxelMorph: A Learning Framework for Deformable Medical Image Registration](https://mlsanity.com/paper/1809.05231/) (2018)
- [Zero-shot Image-to-Image Translation](https://mlsanity.com/paper/2302.03027/) (2023)
- [Label-Efficient Semantic Segmentation with Diffusion Models](https://mlsanity.com/paper/2112.03126/) (2021)
- [Emergent Correspondence from Image Diffusion](https://mlsanity.com/paper/2306.03881/) (2023)
- [DragDiffusion: Harnessing Diffusion Models for Interactive Point-based Image Editing](https://mlsanity.com/paper/2306.14435/) (2023)
- [Unleashing Text-to-Image Diffusion Models for Visual Perception](https://mlsanity.com/paper/2303.02153/) (2023)
- [Diffusion Hyperfeatures: Searching Through Time and Space for Semantic Correspondence](https://mlsanity.com/paper/2305.14334/) (2023)
- [Diffuse, Attend, and Segment: Unsupervised Zero-Shot Segmentation using Stable Diffusion](https://mlsanity.com/paper/2308.12469/) (2023)
- [Denoising Diffusion Autoencoders are Unified Self-supervised Learners](https://mlsanity.com/paper/2303.09769/) (2023)
- [Quicksilver: Fast Predictive Image Registration - a Deep Learning Approach](https://mlsanity.com/paper/1703.10908/) (2017)
- [Deep Learning in Medical Image Registration: A Review](https://mlsanity.com/paper/1912.12318/) (2019)
- [Deconstructing Denoising Diffusion Models for Self-Supervised Learning](https://mlsanity.com/paper/2401.14404/) (2024)
- [uniGradICON: A Foundation Model for Medical Image Registration](https://mlsanity.com/paper/2403.05780/) (2024)
- [Diffusion Models and Representation Learning: A Survey](https://mlsanity.com/paper/2407.00783/) (2024)
- [Exploiting Diffusion Prior for Generalizable Dense Prediction](https://mlsanity.com/paper/2311.18832/) (2023)
- [Learning the Effect of Registration Hyperparameters with HyperMorph](https://mlsanity.com/paper/2203.16680/) (2022)
- [Do text-free diffusion models learn discriminative visual representations?](https://mlsanity.com/paper/2311.17921/) (2023)
- [ICON: Learning Regular Maps Through Inverse Consistency](https://mlsanity.com/paper/2105.04459/) (2021)
- [Not All Diffusion Model Activations Have Been Evaluated as Discriminative Features](https://mlsanity.com/paper/2410.03558/) (2024)
- [FSDiffReg: Feature-wise and Score-wise Diffusion-guided Unsupervised Deformable Image Registration for Cardiac Images](https://mlsanity.com/paper/2307.12035/) (2023)
- [Inverse Consistency by Construction for Multistep Deep Registration](https://mlsanity.com/paper/2305.00087/) (2023)
- [General Purpose Image Encoder DINOv2 for Medical Image Registration](https://mlsanity.com/paper/2402.15687/) (2024)

## Built on by (most cited first, 6 of 6)

- [A 3D Cross-modal Keypoint Descriptor for MR-US Matching and Registration](https://mlsanity.com/paper/2507.18551/) (2025)
- [MedDIFT: Multi-Scale Diffusion-Based Correspondence in 3D Medical Imaging](https://mlsanity.com/paper/2512.05571/) (2025)
- [Flow Matching-enabled Test-Time Refinement for Unsupervised Cardiac MR Registration](https://mlsanity.com/paper/2603.01073/) (2026)
- [CrossFeat: Bridging Imaging Modalities in Feature Descriptor Space](https://mlsanity.com/paper/2609.00272/) (2026)
- [Two-Parameter Flow Map Learning for Continuous-Time Diffeomorphic Image Registration](https://mlsanity.com/paper/2609.10789/) (2026)
- [Benchmarking Intra-Patient 3D Deformable Multimodal Image Registration](https://mlsanity.com/paper/2609.15669/) (2026)

## Abstract (arXiv)

Diffusion models, while trained for image generation, have emerged as powerful foundational feature extractors for downstream tasks. We find that off-the-shelf diffusion models, trained exclusively to generate natural RGB images, can identify semantically meaningful correspondences in medical images. Building on this observation, we propose to leverage diffusion model features as a similarity measure to guide deformable image registration networks. We show that common intensity-based similarity losses often fail in challenging scenarios, such as when certain anatomies are visible in one image but absent in another, leading to anatomically inaccurate alignments. In contrast, our method identifies true semantic correspondences, aligning meaningful structures while disregarding those not present across images. We demonstrate superior performance of our approach on two tasks: multimodal 2D registration (DXA to X-Ray) and monomodal 3D registration (brain-extracted to non-brain-extracted MRI). Code: https://github.com/uncbiag/dgir