# SEKD: Self-Evolving Keypoint Detection and Description

arXiv: https://arxiv.org/abs/2006.05077 · HTML: https://mlsanity.com/paper/2006.05077/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Jun 2020.

## Built on by (most cited first, 11 of 11)

- [ALIKED: A Lighter Keypoint and Descriptor Extraction Network via Deformable Transformation](https://mlsanity.com/paper/2304.03608/) (2023)
- [ALIKE: Accurate and Lightweight Keypoint Detection and Descriptor Extraction](https://mlsanity.com/paper/2112.02906/) (2021)
- [SOLD2: Self-supervised Occlusion-aware Line Description and Detection](https://mlsanity.com/paper/2104.03362/) (2021)
- [Global Multi-modal 2D/3D Registration via Local Descriptors Learning](https://mlsanity.com/paper/2205.03439/) (2022)
- [Deep Learning Reforms Image Matching: A Survey and Outlook](https://mlsanity.com/paper/2506.04619/) (2025)
- [Learning to Detect Good Keypoints to Match Non-Rigid Objects in RGB Images](https://mlsanity.com/paper/2212.09589/) (2022)
- [Discriminative and Semantic Feature Selection for Place Recognition towards Dynamic Environments](https://mlsanity.com/paper/2103.10023/) (2021)
- [Improving the matching of deformable objects by learning to detect keypoints](https://mlsanity.com/paper/2309.00434/) (2023)
- [Entropy-driven Unsupervised Keypoint Representation Learning in Videos](https://mlsanity.com/paper/2209.15404/) (2022)
- [Real-time Local Feature with Global Visual Information Enhancement](https://mlsanity.com/paper/2211.10981/) (2022)
- [Second-order Anisotropic Gaussian Directional Derivative Filters for Blob Detection](https://mlsanity.com/paper/2305.00435/) (2023)

## Abstract (arXiv)

Researchers have attempted utilizing deep neural network (DNN) to learn novel local features from images inspired by its recent successes on a variety of vision tasks. However, existing DNN-based algorithms have not achieved such remarkable progress that could be partly attributed to insufficient utilization of the interactive characters between local feature detector and descriptor. To alleviate these difficulties, we emphasize two desired properties, i.e., repeatability and reliability, to simultaneously summarize the inherent and interactive characters of local feature detector and descriptor. Guided by these properties, a self-supervised framework, namely self-evolving keypoint detection and description (SEKD), is proposed to learn an advanced local feature model from unlabeled natural images. Additionally, to have performance guarantees, novel training strategies have also been dedicatedly designed to minimize the gap between the learned feature and its properties. We benchmark the proposed method on homography estimation, relative pose estimation, and structure-from-motion tasks. Extensive experimental results demonstrate that the proposed method outperforms popular hand-crafted and DNN-based methods by remarkable margins. Ablation studies also verify the effectiveness of each critical training strategy. We will release our code along with the trained model publicly.