# Self-supervised Interest Point Detection and Description for Fisheye and Perspective Images

arXiv: https://arxiv.org/abs/2306.01938 · HTML: https://mlsanity.com/paper/2306.01938/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 39 references that are in the map.
- First posted on arXiv 2 Jun 2023.

## Builds on (most cited first, 15 of 15)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D](https://mlsanity.com/paper/2109.13410/) (2021)
- [LIFT: Learned Invariant Feature Transform](https://mlsanity.com/paper/1603.09114/) (2016)
- [R2D2: Repeatable and Reliable Detector and Descriptor](https://mlsanity.com/paper/1906.06195/) (2019)
- [LF-Net: Learning Local Features from Images](https://mlsanity.com/paper/1805.09662/) (2018)
- [TILDE: A Temporally Invariant Learned DEtector](https://mlsanity.com/paper/1411.4568/) (2014)
- [Learning Covariant Feature Detectors](https://mlsanity.com/paper/1605.01224/) (2016)
- [RoRD: Rotation-Robust Descriptors and Orthographic Views for Local Feature Matching](https://mlsanity.com/paper/2103.08573/) (2021)
- [Learning to Assign Orientations to Feature Points](https://mlsanity.com/paper/1511.04273/) (2015)
- [G2D: from GTA to Data](https://mlsanity.com/paper/1806.07381/) (2018)
- [3D Reconstruction from Full-view Fisheye Camera](https://mlsanity.com/paper/1506.06273/) (2015)
- [FisheyeSuperPoint: Keypoint Detection and Description Network for Fisheye Images](https://mlsanity.com/paper/2103.00191/) (2021)

## Built on by (most cited first, 3 of 3)

- [RegWSI: Whole Slide Image Registration using Combined Deep Feature- and Intensity-Based Methods: Winner of the ACROBAT 2023 Challenge](https://mlsanity.com/paper/2404.13108/) (2024)
- [Learning Probabilistic Coordinate Fields for Robust Correspondences](https://mlsanity.com/paper/2306.04231/) (2023)
- [CodingHomo: Bootstrapping Deep Homography With Video Coding](https://mlsanity.com/paper/2504.12165/) (2025)

## Abstract (arXiv)

Keypoint detection and matching is a fundamental task in many computer vision problems, from shape reconstruction, to structure from motion, to AR/VR applications and robotics. It is a well-studied problem with remarkable successes such as SIFT, and more recent deep learning approaches. While great robustness is exhibited by these techniques with respect to noise, illumination variation, and rigid motion transformations, less attention has been placed on image distortion sensitivity. In this work, we focus on the case when this is caused by the geometry of the cameras used for image acquisition, and consider the keypoint detection and matching problem between the hybrid scenario of a fisheye and a projective image. We build on a state-of-the-art approach and derive a self-supervised procedure that enables training an interest point detector and descriptor network. We also collected two new datasets for additional training and testing in this unexplored scenario, and we demonstrate that current approaches are suboptimal because they are designed to work in traditional projective conditions, while the proposed approach turns out to be the most effective.