# Retinal IPA: Iterative KeyPoints Alignment for Multimodal Retinal Imaging

arXiv: https://arxiv.org/abs/2407.18362 · HTML: https://mlsanity.com/paper/2407.18362/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 32 references that are in the map.
- First posted on arXiv 25 Jul 2024.

## Builds on (most cited first, 20 of 20)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [LoFTR: Detector-Free Local Feature Matching with Transformers](https://mlsanity.com/paper/2104.00680/) (2021)
- [LightGlue: Local Feature Matching at Light Speed](https://mlsanity.com/paper/2306.13643/) (2023)
- [DISK: Learning local features with policy gradient](https://mlsanity.com/paper/2006.13566/) (2020)
- [R2D2: Repeatable and Reliable Detector and Descriptor](https://mlsanity.com/paper/1906.06195/) (2019)
- [DKM: Dense Kernelized Feature Matching for Geometry Estimation](https://mlsanity.com/paper/2202.00667/) (2022)
- [ASpanFormer: Detector-Free Image Matching with Adaptive Span Transformer](https://mlsanity.com/paper/2208.14201/) (2022)
- [Positional Contrastive Learning for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/2106.09157/) (2021)
- [SiLK -- Simple Learned Keypoints](https://mlsanity.com/paper/2304.06194/) (2023)
- [OCTA-500: A Retinal Dataset for Optical Coherence Tomography Angiography Study](https://mlsanity.com/paper/2012.07261/) (2020)
- [GLAMpoints: Greedily Learned Accurate Match points](https://mlsanity.com/paper/1908.06812/) (2019)
- [Semi-Supervised Keypoint Detector and Descriptor for Retinal Image Matching](https://mlsanity.com/paper/2207.07932/) (2022)
- [Promise:Prompt-driven 3D Medical Image Segmentation Using Pretrained Image Foundation Models](https://mlsanity.com/paper/2310.19721/) (2023)
- [Reverse Knowledge Distillation: Training a Large Model using a Small One for Retinal Image Matching on Limited Data](https://mlsanity.com/paper/2307.10698/) (2023)
- [Multi-modal Retinal Image Registration Using a Keypoint-Based Vessel Structure Aligning Network](https://mlsanity.com/paper/2207.10506/) (2022)
- [Keypoint-Augmented Self-Supervised Learning for Medical Image Segmentation with Limited Annotation](https://mlsanity.com/paper/2310.01680/) (2023)
- [Novel OCT mosaicking pipeline with Feature- and Pixel-based registration](https://mlsanity.com/paper/2311.13052/) (2023)

## Built on by (most cited first, 3 of 3)

- [Unsupervised Deformable Image Registration with Structural Nonparametric Smoothing](https://mlsanity.com/paper/2506.10813/) (2025)
- [SynStitch: a Self-Supervised Learning Network for Ultrasound Image Stitching Using Synthetic Training Pairs and Indirect Supervision](https://mlsanity.com/paper/2411.06750/) (2024)
- [Fidelity-Imposed Displacement Editing for the Learn2Reg 2024 SHG-BF Challenge](https://mlsanity.com/paper/2410.20812/) (2024)

## Abstract (arXiv)

We propose a novel framework for retinal feature point alignment, designed for learning cross-modality features to enhance matching and registration across multi-modality retinal images. Our model draws on the success of previous learning-based feature detection and description methods. To better leverage unlabeled data and constrain the model to reproduce relevant keypoints, we integrate a keypoint-based segmentation task. It is trained in a self-supervised manner by enforcing segmentation consistency between different augmentations of the same image. By incorporating a keypoint augmented self-supervised layer, we achieve robust feature extraction across modalities. Extensive evaluation on two public datasets and one in-house dataset demonstrates significant improvements in performance for modality-agnostic retinal feature alignment. Our code and model weights are publicly available at urlhttps://github.com/MedICL-VU/RetinaIPA.