# Learning Covariant Feature Detectors

arXiv: https://arxiv.org/abs/1605.01224 · HTML: https://mlsanity.com/paper/1605.01224/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 May 2016.

## Built on by (most cited first, 25 of 25)

- [BALF: Simple and Efficient Blur Aware Local Feature Detector](https://mlsanity.com/paper/2211.14731/) (2022)
- [A case for using rotation invariant features in state of the art feature matchers](https://mlsanity.com/paper/2204.10144/) (2022)
- [Self-Supervised Equivariant Learning for Oriented Keypoint Detection](https://mlsanity.com/paper/2204.08613/) (2022)
- [Matching 2D Images in 3D: Metric Relative Pose from Metric Correspondences](https://mlsanity.com/paper/2404.06337/) (2024)
- [A Robust and Interpretable Deep Learning Framework for Multi-modal Registration via Keypoints](https://mlsanity.com/paper/2304.09941/) (2023)
- [Partially Does It: Towards Scene-Level FG-SBIR with Partial Input](https://mlsanity.com/paper/2203.14804/) (2022)
- [DFM: A Performance Baseline for Deep Feature Matching](https://mlsanity.com/paper/2106.07791/) (2021)
- [DenseGAP: Graph-Structured Dense Correspondence Learning with Anchor Points](https://mlsanity.com/paper/2112.06910/) (2021)
- [A Critical Analysis of Image-based Camera Pose Estimation Techniques](https://mlsanity.com/paper/2201.05816/) (2022)
- [Deep Learning Reforms Image Matching: A Survey and Outlook](https://mlsanity.com/paper/2506.04619/) (2025)
- [Anomaly Detection of Defect using Energy of Point Pattern Features within Random Finite Set Framework](https://mlsanity.com/paper/2108.12159/) (2021)
- [TUSK: Task-Agnostic Unsupervised Keypoints](https://mlsanity.com/paper/2206.08460/) (2022)
- [Self-Supervised Correspondence Estimation via Multiview Registration](https://mlsanity.com/paper/2212.03236/) (2022)
- [From Keypoints to Object Landmarks via Self-Training Correspondence: A novel approach to Unsupervised Landmark Discovery](https://mlsanity.com/paper/2205.15895/) (2022)
- [Digging Into Self-Supervised Learning of Feature Descriptors](https://mlsanity.com/paper/2110.04773/) (2021)
- [Self-supervised Interest Point Detection and Description for Fisheye and Perspective Images](https://mlsanity.com/paper/2306.01938/) (2023)
- [Image Feature Information Extraction for Interest Point Detection: A Review](https://mlsanity.com/paper/2106.07929/) (2021)
- [Soft Expectation and Deep Maximization for Image Feature Detection](https://mlsanity.com/paper/2104.10291/) (2021)
- [Transformation Coding: Simple Objectives for Equivariant Representations](https://mlsanity.com/paper/2202.10930/) (2022)
- [Restore Translation Using Equivariant Neural Networks](https://mlsanity.com/paper/2306.16938/) (2023)
- [Residual Learning for Image Point Descriptors](https://mlsanity.com/paper/2312.15471/) (2023)
- [Synergistic eigenanalysis of covariance and Hessian matrices for enhanced binary classification](https://mlsanity.com/paper/2402.09281/) (2024)
- [Keypoint Aware Masked Image Modelling](https://mlsanity.com/paper/2407.13873/) (2024)
- [RealKeyMorph: Keypoints in Real-world Coordinates for Resolution-agnostic Image Registration](https://mlsanity.com/paper/2506.10344/) (2025)
- [FPC-Net: Revisiting SuperPoint with Descriptor-Free Keypoint Detection via Feature Pyramids and Consistency-Based Implicit Matching](https://mlsanity.com/paper/2507.10770/) (2025)

## Abstract (arXiv)

Local covariant feature detection, namely the problem of extracting viewpoint invariant features from images, has so far largely resisted the application of machine learning techniques. In this paper, we propose the first fully general formulation for learning local covariant feature detectors. We propose to cast detection as a regression problem, enabling the use of powerful regressors such as deep neural networks. We then derive a covariance constraint that can be used to automatically learn which visual structures provide stable anchors for local feature detection. We support these ideas theoretically, proposing a novel analysis of local features in term of geometric transformations, and we show that all common and many uncommon detectors can be derived in this framework. Finally, we present empirical results on translation and rotation covariant detectors on standard feature benchmarks, showing the power and flexibility of the framework.