# SiLK -- Simple Learned Keypoints

arXiv: https://arxiv.org/abs/2304.06194 · HTML: https://mlsanity.com/paper/2304.06194/

## Key facts

- Cited by 54 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 60 references that are in the map.
- First posted on arXiv 12 Apr 2023.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [LoFTR: Detector-Free Local Feature Matching with Transformers](https://mlsanity.com/paper/2104.00680/) (2021)
- [MegaDepth: Learning Single-View Depth Prediction from Internet Photos](https://mlsanity.com/paper/1804.00607/) (2018)
- [Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstruction](https://mlsanity.com/paper/2109.00512/) (2021)
- [DISK: Learning local features with policy gradient](https://mlsanity.com/paper/2006.13566/) (2020)
- [HPatches: A benchmark and evaluation of handcrafted and learned local descriptors](https://mlsanity.com/paper/1704.05939/) (2017)
- [Deep Global Registration](https://mlsanity.com/paper/2004.11540/) (2020)
- [Image Matching across Wide Baselines: From Paper to Practice](https://mlsanity.com/paper/2003.01587/) (2020)
- [R2D2: Repeatable and Reliable Detector and Descriptor](https://mlsanity.com/paper/1906.06195/) (2019)
- [Learning to Find Good Correspondences](https://mlsanity.com/paper/1711.05971/) (2017)
- [Self-attention Does Not Need O(n2) Memory](https://mlsanity.com/paper/2112.05682/) (2021)
- [Learning Two-View Correspondences and Geometry Using Order-Aware Network](https://mlsanity.com/paper/1908.04964/) (2019)
- [Neural-Guided RANSAC: Learning Where to Sample Model Hypotheses](https://mlsanity.com/paper/1905.04132/) (2019)
- [MAGSAC: marginalizing sample consensus](https://mlsanity.com/paper/1803.07469/) (2018)
- [Dual-Resolution Correspondence Networks](https://mlsanity.com/paper/2006.08844/) (2020)
- [Learning multiview 3D point cloud registration](https://mlsanity.com/paper/2001.05119/) (2020)
- [UnsupervisedR&R: Unsupervised Point Cloud Registration via Differentiable Rendering](https://mlsanity.com/paper/2102.11870/) (2021)
- [GLAMpoints: Greedily Learned Accurate Match points](https://mlsanity.com/paper/1908.06812/) (2019)
- [UnsuperPoint: End-to-end Unsupervised Interest Point Detector and Descriptor](https://mlsanity.com/paper/1907.04011/) (2019)
- [Self-supervised Keypoint Correspondences for Multi-Person Pose Estimation and Tracking in Videos](https://mlsanity.com/paper/2004.12652/) (2020)
- [Neural Outlier Rejection for Self-Supervised Keypoint Learning](https://mlsanity.com/paper/1912.10615/) (2019)

## Built on by (most cited first, 30 of 54)

- [Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Pass](https://mlsanity.com/paper/2501.13928/) (2025)
- [RoMa: Robust Dense Feature Matching](https://mlsanity.com/paper/2305.15404/) (2023)
- [XFeat: Accelerated Features for Lightweight Image Matching](https://mlsanity.com/paper/2404.19174/) (2024)
- [DeDoDe: Detect, Don't Describe -- Describe, Don't Detect for Local Feature Matching](https://mlsanity.com/paper/2308.08479/) (2023)
- [DeDoDe v2: Analyzing and Improving the DeDoDe Keypoint Detector](https://mlsanity.com/paper/2404.08928/) (2024)
- [Matching 2D Images in 3D: Metric Relative Pose from Metric Correspondences](https://mlsanity.com/paper/2404.06337/) (2024)
- [Steerers: A framework for rotation equivariant keypoint descriptors](https://mlsanity.com/paper/2312.02152/) (2023)
- [MESA: Matching Everything by Segmenting Anything](https://mlsanity.com/paper/2401.16741/) (2024)
- [Segment Anything Model is a Good Teacher for Local Feature Learning](https://mlsanity.com/paper/2309.16992/) (2023)
- [EarthMatch: Iterative Coregistration for Fine-grained Localization of Astronaut Photography](https://mlsanity.com/paper/2405.05422/) (2024)
- [RDD: Robust Feature Detector and Descriptor using Deformable Transformer](https://mlsanity.com/paper/2505.08013/) (2025)
- [LiftFeat: 3D Geometry-Aware Local Feature Matching](https://mlsanity.com/paper/2505.03422/) (2025)
- [Alligat0R: Pre-Training Through Co-Visibility Segmentation for Relative Camera Pose Regression](https://mlsanity.com/paper/2503.07561/) (2025)
- [Regist3R: Incremental Registration with Stereo Foundation Model](https://mlsanity.com/paper/2504.12356/) (2025)
- [RIPE: Reinforcement Learning on Unlabeled Image Pairs for Robust Keypoint Extraction](https://mlsanity.com/paper/2507.04839/) (2025)
- [RUBIK: A Structured Benchmark for Image Matching across Geometric Challenges](https://mlsanity.com/paper/2502.19955/) (2025)
- [Raising the Ceiling: Conflict-Free Local Feature Matching with Dynamic View Switching](https://mlsanity.com/paper/2407.07789/) (2024)
- [EDM: Efficient Deep Feature Matching](https://mlsanity.com/paper/2503.05122/) (2025)
- [Affine steerers for structured keypoint description](https://mlsanity.com/paper/2408.14186/) (2024)
- [MIFNet: Learning Modality-Invariant Features for Generalizable Multimodal Image Matching](https://mlsanity.com/paper/2501.11299/) (2025)
- [SuperEvent: Cross-Modal Learning of Event-based Keypoint Detection for SLAM](https://mlsanity.com/paper/2504.00139/) (2025)
- [FocusFlow: Boosting Key-Points Optical Flow Estimation for Autonomous Driving](https://mlsanity.com/paper/2308.07104/) (2023)
- [RIDE: Self-Supervised Learning of Rotation-Equivariant Keypoint Detection and Invariant Description for Endoscopy](https://mlsanity.com/paper/2309.09563/) (2023)
- [Pose-Guided Self-Training with Two-Stage Clustering for Unsupervised Landmark Discovery](https://mlsanity.com/paper/2403.16194/) (2024)
- [Retinal IPA: Iterative KeyPoints Alignment for Multimodal Retinal Imaging](https://mlsanity.com/paper/2407.18362/) (2024)
- [MESA: Effective Matching Redundancy Reduction by Semantic Area Segmentation](https://mlsanity.com/paper/2408.00279/) (2024)
- [ADen: Adaptive Density Representations for Sparse-view Camera Pose Estimation](https://mlsanity.com/paper/2408.09042/) (2024)
- [To Match or Not to Match: Revisiting Image Matching for Reliable Visual Place Recognition](https://mlsanity.com/paper/2504.06116/) (2025)
- [Unsupervised Deformable Image Registration with Structural Nonparametric Smoothing](https://mlsanity.com/paper/2506.10813/) (2025)
- [RaCo: Ranking and Covariance for Practical Learned Keypoints](https://mlsanity.com/paper/2602.15755/) (2026)

## Abstract (arXiv)

Keypoint detection & descriptors are foundational tech-nologies for computer vision tasks like image matching, 3D reconstruction and visual odometry. Hand-engineered methods like Harris corners, SIFT, and HOG descriptors have been used for decades; more recently, there has been a trend to introduce learning in an attempt to improve keypoint detectors. On inspection however, the results are difficult to interpret; recent learning-based methods employ a vast diversity of experimental setups and design choices: empirical results are often reported using different backbones, protocols, datasets, types of supervisions or tasks. Since these differences are often coupled together, it raises a natural question on what makes a good learned keypoint detector. In this work, we revisit the design of existing keypoint detectors by deconstructing their methodologies and identifying the key components. We re-design each component from first-principle and propose Simple Learned Keypoints (SiLK) that is fully-differentiable, lightweight, and flexible. Despite its simplicity, SiLK advances new state-of-the-art on Detection Repeatability and Homography Estimation tasks on HPatches and 3D Point-Cloud Registration task on ScanNet, and achieves competitive performance to state-of-the-art on camera pose estimation in 2022 Image Matching Challenge and ScanNet.