# RetinaRegNet: A Zero-Shot Approach for Retinal Image Registration

arXiv: https://arxiv.org/abs/2404.16017 · HTML: https://mlsanity.com/paper/2404.16017/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 73 references that are in the map.
- First posted on arXiv 24 Apr 2024.

## Builds on (most cited first, 28 of 28)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [Null-text Inversion for Editing Real Images using Guided Diffusion Models](https://mlsanity.com/paper/2211.09794/) (2022)
- [Variational Diffusion Models](https://mlsanity.com/paper/2107.00630/) (2021)
- [LoFTR: Detector-Free Local Feature Matching with Transformers](https://mlsanity.com/paper/2104.00680/) (2021)
- [Emergent Correspondence from Image Diffusion](https://mlsanity.com/paper/2306.03881/) (2023)
- [SRDiff: Single Image Super-Resolution with Diffusion Probabilistic Models](https://mlsanity.com/paper/2104.14951/) (2021)
- [SegDiff: Image Segmentation with Diffusion Probabilistic Models](https://mlsanity.com/paper/2112.00390/) (2021)
- [Diffusion Models for Medical Image Analysis: A Comprehensive Survey](https://mlsanity.com/paper/2211.07804/) (2022)
- [Neighbourhood Consensus Networks](https://mlsanity.com/paper/1810.10510/) (2018)
- [ASpanFormer: Detector-Free Image Matching with Adaptive Span Transformer](https://mlsanity.com/paper/2208.14201/) (2022)
- [Unsupervised Semantic Correspondence Using Stable Diffusion](https://mlsanity.com/paper/2305.15581/) (2023)
- [Diffusion Model as Representation Learner](https://mlsanity.com/paper/2308.10916/) (2023)
- [PET image denoising based on denoising diffusion probabilistic models](https://mlsanity.com/paper/2209.06167/) (2022)
- [SCNet: Learning Semantic Correspondence](https://mlsanity.com/paper/1705.04043/) (2017)
- [Multi-scale Matching Networks for Semantic Correspondence](https://mlsanity.com/paper/2108.00211/) (2021)
- [Semi-Supervised Keypoint Detector and Descriptor for Retinal Image Matching](https://mlsanity.com/paper/2207.07932/) (2022)
- [Reverse Knowledge Distillation: Training a Large Model using a Small One for Retinal Image Matching on Limited Data](https://mlsanity.com/paper/2307.10698/) (2023)
- [ProsRegNet: A Deep Learning Framework for Registration of MRI and Histopathology Images of the Prostate](https://mlsanity.com/paper/2012.00991/) (2020)
- [Geodesic Density Regression for Correcting 4DCT Pulmonary Respiratory Motion Artifacts](https://mlsanity.com/paper/2106.06853/) (2021)
- [MEMO: Dataset and Methods for Robust Multimodal Retinal Image Registration with Large or Small Vessel Density Differences](https://mlsanity.com/paper/2309.14550/) (2023)
- [Privacy Threats in Stable Diffusion Models](https://mlsanity.com/paper/2311.09355/) (2023)

## Built on by (most cited first, 7 of 7)

- [Auto-regressive transformation for image alignment](https://mlsanity.com/paper/2505.04864/) (2025)
- [Gaussian Primitive Optimized Deformable Retinal Image Registration](https://mlsanity.com/paper/2508.16852/) (2025)
- [Establishing Robust Retinal Eye Tracking: A Weakly Supervised Algorithmic Framework](https://mlsanity.com/paper/2605.09181/) (2026)
- [Unsupervised training of keypoint-agnostic descriptors for flexible retinal image registration](https://mlsanity.com/paper/2505.02787/) (2025)
- [MatRes: Zero-Shot Test-Time Model Adaptation for Simultaneous Matching and Restoration](https://mlsanity.com/paper/2604.10081/) (2026)
- [Particle Diffusion Matching: Random Walk Correspondence Search for the Alignment of Standard and Ultra-Widefield Fundus Images](https://mlsanity.com/paper/2604.10085/) (2026)
- [Modality-Invariant Coarse-to-Fine Retinal Image Registration](https://mlsanity.com/paper/2608.14829/) (2026)

## Abstract (arXiv)

We introduce RetinaRegNet, a zero-shot image registration model designed to register retinal images with minimal overlap, large deformations, and varying image quality. RetinaRegNet addresses these challenges and achieves robust and accurate registration through the following steps. First, we extract features from the moving and fixed images using latent diffusion models. We then sample feature points from the fixed image using a combination of the SIFT algorithm and random point sampling. For each sampled point, we identify its corresponding point in the moving image using a 2D correlation map, which computes the cosine similarity between the diffusion feature vectors of the point in the fixed image and all pixels in the moving image. Second, we eliminate most incorrectly detected point correspondences (outliers) by enforcing an inverse consistency constraint, ensuring that correspondences are consistent in both forward and backward directions. We further remove outliers with large distances between corresponding points using a global transformation based outlier detector. Finally, we implement a two-stage registration framework to handle large deformations. The first stage estimates a homography transformation to achieve global alignment between the images, while the second stage uses a third-order polynomial transformation to estimate local deformations. We evaluated RetinaRegNet on three retinal image registration datasets: color fundus images, fluorescein angiography images, and laser speckle flowgraphy images. Our model consistently outperformed state-of-the-art methods across all datasets. The accurate registration achieved by RetinaRegNet enables the tracking of eye disease progression, enhances surgical planning, and facilitates the evaluation of treatment efficacy. Our code is publicly available at: https://github.com/mirthAI/RetinaRegNet.