# Self-Supervised Equivariant Learning for Oriented Keypoint Detection

arXiv: https://arxiv.org/abs/2204.08613 · HTML: https://mlsanity.com/paper/2204.08613/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 48 of its 79 references that are in the map.
- First posted on arXiv 19 Apr 2022.

## Builds on (most cited first, 30 of 48)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016)
- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [General E(2)-Equivariant Steerable CNNs](https://mlsanity.com/paper/1911.08251/) (2019)
- [D2-Net: A Trainable CNN for Joint Detection and Description of Local Features](https://mlsanity.com/paper/1905.03561/) (2019)
- [DISK: Learning local features with policy gradient](https://mlsanity.com/paper/2006.13566/) (2020)
- [LIFT: Learned Invariant Feature Transform](https://mlsanity.com/paper/1603.09114/) (2016)
- [Steerable CNNs](https://mlsanity.com/paper/1612.08498/) (2016)
- [Harmonic Networks: Deep Translation and Rotation Equivariance](https://mlsanity.com/paper/1612.04642/) (2016)
- [HPatches: A benchmark and evaluation of handcrafted and learned local descriptors](https://mlsanity.com/paper/1704.05939/) (2017)
- [Large-Scale Image Retrieval with Attentive Deep Local Features](https://mlsanity.com/paper/1612.06321/) (2016)
- [Kornia: an Open Source Differentiable Computer Vision Library for PyTorch](https://mlsanity.com/paper/1910.02190/) (2019)
- [A General Theory of Equivariant CNNs on Homogeneous Spaces](https://mlsanity.com/paper/1811.02017/) (2018)
- [Image Matching across Wide Baselines: From Paper to Practice](https://mlsanity.com/paper/2003.01587/) (2020)
- [Working hard to know your neighbor's margins: Local descriptor learning loss](https://mlsanity.com/paper/1705.10872/) (2017)
- [ReDet: A Rotation-equivariant Detector for Aerial Object Detection](https://mlsanity.com/paper/2103.07733/) (2021)
- [Learning Steerable Filters for Rotation Equivariant CNNs](https://mlsanity.com/paper/1711.07289/) (2017)
- [Learning to Find Good Correspondences](https://mlsanity.com/paper/1711.05971/) (2017)
- [Neighbourhood Consensus Networks](https://mlsanity.com/paper/1810.10510/) (2018)
- [LF-Net: Learning Local Features from Images](https://mlsanity.com/paper/1805.09662/) (2018)
- [Learning Two-View Correspondences and Geometry Using Order-Aware Network](https://mlsanity.com/paper/1908.04964/) (2019)
- [GLU-Net: Global-Local Universal Network for Dense Flow and Correspondences](https://mlsanity.com/paper/1912.05524/) (2019)
- [SOSNet: Second Order Similarity Regularization for Local Descriptor Learning](https://mlsanity.com/paper/1904.05019/) (2019)
- [Neural-Guided RANSAC: Learning Where to Sample Model Hypotheses](https://mlsanity.com/paper/1905.04132/) (2019)
- [Key.Net: Keypoint Detection by Handcrafted and Learned CNN Filters](https://mlsanity.com/paper/1904.00889/) (2019)
- [Scale-Equivariant Steerable Networks](https://mlsanity.com/paper/1910.11093/) (2019)

## Built on by (most cited first, 26 of 26)

- [6DGS: 6D Pose Estimation from a Single Image and a 3D Gaussian Splatting Model](https://mlsanity.com/paper/2407.15484/) (2024)
- [Self-supervised remote sensing feature learning: Learning Paradigms, Challenges, and Future Works](https://mlsanity.com/paper/2211.08129/) (2022)
- [Learning Rotation-Equivariant Features for Visual Correspondence](https://mlsanity.com/paper/2303.15472/) (2023)
- [Steerers: A framework for rotation equivariant keypoint descriptors](https://mlsanity.com/paper/2312.02152/) (2023)
- [E2PN: Efficient SE(3)-Equivariant Point Network](https://mlsanity.com/paper/2206.05398/) (2022)
- [S-TREK: Sequential Translation and Rotation Equivariant Keypoints for local feature extraction](https://mlsanity.com/paper/2308.14598/) (2023)
- [Learning to Make Keypoints Sub-Pixel Accurate](https://mlsanity.com/paper/2407.11668/) (2024)
- [DaD: Distilled Reinforcement Learning for Diverse Keypoint Detection](https://mlsanity.com/paper/2503.07347/) (2025)
- [Searching from Area to Point: A Hierarchical Framework for Semantic-Geometric Combined Feature Matching](https://mlsanity.com/paper/2305.00194/) (2023)
- [Deep Learning Reforms Image Matching: A Survey and Outlook](https://mlsanity.com/paper/2506.04619/) (2025)
- [Affine steerers for structured keypoint description](https://mlsanity.com/paper/2408.14186/) (2024)
- [NeSS-ST: Detecting Good and Stable Keypoints with a Neural Stability Score and the Shi-Tomasi Detector](https://mlsanity.com/paper/2307.01069/) (2023)
- [RIDE: Self-Supervised Learning of Rotation-Equivariant Keypoint Detection and Invariant Description for Endoscopy](https://mlsanity.com/paper/2309.09563/) (2023)
- [Incorporating Arbitrary Matrix Group Equivariance into KANs](https://mlsanity.com/paper/2410.00435/) (2024)
- [3D Equivariant Pose Regression via Direct Wigner-D Harmonics Prediction](https://mlsanity.com/paper/2411.00543/) (2024)
- [SplatPose: Geometry-Aware 6-DoF Pose Estimation from Single RGB Image via 3D Gaussian Splatting](https://mlsanity.com/paper/2503.05174/) (2025)
- [Second-order Gaussian directional derivative representations for image high-resolution corner detection](https://mlsanity.com/paper/2601.08182/) (2026)
- [RaCo: Ranking and Covariance for Practical Learned Keypoints](https://mlsanity.com/paper/2602.15755/) (2026)
- [Who Handles Orientation? Investigating Invariance in Feature Matching](https://mlsanity.com/paper/2604.11809/) (2026)
- [REViT: Roto-reflection Equivariant Convolutional Vision Transformer](https://mlsanity.com/paper/2606.25318/) (2026)
- [SIM2E: Benchmarking the Group Equivariant Capability of Correspondence Matching Algorithms](https://mlsanity.com/paper/2208.09896/) (2022)
- [Residual Learning for Image Point Descriptors](https://mlsanity.com/paper/2312.15471/) (2023)
- [Good Keypoints for the Two-View Geometry Estimation Problem](https://mlsanity.com/paper/2503.18767/) (2025)
- [Equivariant symmetry-aware head pose estimation for fetal MRI](https://mlsanity.com/paper/2512.04890/) (2025)
- [From Pairs to Sequences: Track-Aware Policy Gradients for Keypoint Detection](https://mlsanity.com/paper/2602.20630/) (2026)
- [SSMB: Self-Supervised Local Feature Detection under Motion Blur](https://mlsanity.com/paper/2608.27181/) (2026)

## Abstract (arXiv)

Detecting robust keypoints from an image is an integral part of many computer vision problems, and the characteristic orientation and scale of keypoints play an important role for keypoint description and matching. Existing learning-based methods for keypoint detection rely on standard translation-equivariant CNNs but often fail to detect reliable keypoints against geometric variations. To learn to detect robust oriented keypoints, we introduce a self-supervised learning framework using rotation-equivariant CNNs. We propose a dense orientation alignment loss by an image pair generated by synthetic transformations for training a histogram-based orientation map. Our method outperforms the previous methods on an image matching benchmark and a camera pose estimation benchmark.