# DaD: Distilled Reinforcement Learning for Diverse Keypoint Detection

arXiv: https://arxiv.org/abs/2503.07347 · HTML: https://mlsanity.com/paper/2503.07347/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods.
- Builds on 31 of its 48 references that are in the map.
- First posted on arXiv 10 Mar 2025.

## Placed in

- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 30 of 31)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [LoFTR: Detector-Free Local Feature Matching with Transformers](https://mlsanity.com/paper/2104.00680/) (2021)
- [LightGlue: Local Feature Matching at Light Speed](https://mlsanity.com/paper/2306.13643/) (2023)
- [DISK: Learning local features with policy gradient](https://mlsanity.com/paper/2006.13566/) (2020)
- [RoMa: Robust Dense Feature Matching](https://mlsanity.com/paper/2305.15404/) (2023)
- [DSAC - Differentiable RANSAC for Camera Localization](https://mlsanity.com/paper/1611.05705/) (2016)
- [HPatches: A benchmark and evaluation of handcrafted and learned local descriptors](https://mlsanity.com/paper/1704.05939/) (2017)
- [Global Structure-from-Motion Revisited](https://mlsanity.com/paper/2407.20219/) (2024)
- [DKM: Dense Kernelized Feature Matching for Geometry Estimation](https://mlsanity.com/paper/2202.00667/) (2022)
- [Neural-Guided RANSAC: Learning Where to Sample Model Hypotheses](https://mlsanity.com/paper/1905.04132/) (2019)
- [ALIKED: A Lighter Keypoint and Descriptor Extraction Network via Deformable Transformation](https://mlsanity.com/paper/2304.03608/) (2023)
- [Efficient LoFTR: Semi-Dense Local Feature Matching with Sparse-Like Speed](https://mlsanity.com/paper/2403.04765/) (2024)
- [Key.Net: Keypoint Detection by Handcrafted and Learned CNN Filters](https://mlsanity.com/paper/1904.00889/) (2019)
- [PDC-Net+: Enhanced Probabilistic Dense Correspondence Network](https://mlsanity.com/paper/2109.13912/) (2021)
- [ALIKE: Accurate and Lightweight Keypoint Detection and Descriptor Extraction](https://mlsanity.com/paper/2112.02906/) (2021)
- [Detector-Free Structure from Motion](https://mlsanity.com/paper/2306.15669/) (2023)
- [DeDoDe: Detect, Don't Describe -- Describe, Don't Detect for Local Feature Matching](https://mlsanity.com/paper/2308.08479/) (2023)
- [Reinforced Feature Points: Optimizing Feature Detection and Description for a High-Level Task](https://mlsanity.com/paper/1912.00623/) (2019)
- [DeDoDe v2: Analyzing and Improving the DeDoDe Keypoint Detector](https://mlsanity.com/paper/2404.08928/) (2024)
- [Generalized Differentiable RANSAC](https://mlsanity.com/paper/2212.13185/) (2022)
- [Decoupling Makes Weakly Supervised Local Feature Better](https://mlsanity.com/paper/2201.02861/) (2022)
- [Self-Supervised Equivariant Learning for Oriented Keypoint Detection](https://mlsanity.com/paper/2204.08613/) (2022)
- [Steerers: A framework for rotation equivariant keypoint descriptors](https://mlsanity.com/paper/2312.02152/) (2023)
- [S-TREK: Sequential Translation and Rotation Equivariant Keypoints for local feature extraction](https://mlsanity.com/paper/2308.14598/) (2023)
- [Color Equivariant Convolutional Networks](https://mlsanity.com/paper/2310.19368/) (2023)
- [Affine steerers for structured keypoint description](https://mlsanity.com/paper/2408.14186/) (2024)
- [Learning Color Equivariant Representations](https://mlsanity.com/paper/2406.09588/) (2024)

## Built on by (most cited first, 13 of 13)

- [LoMa: Local Feature Matching Revisited](https://mlsanity.com/paper/2604.04931/) (2026)
- [Deep Learning Reforms Image Matching: A Survey and Outlook](https://mlsanity.com/paper/2506.04619/) (2025)
- [RaCo: Ranking and Covariance for Practical Learned Keypoints](https://mlsanity.com/paper/2602.15755/) (2026)
- [Understanding and Optimizing Attention-Based Sparse Matching for Diverse Local Features](https://mlsanity.com/paper/2602.08430/) (2026)
- [Who Handles Orientation? Investigating Invariance in Feature Matching](https://mlsanity.com/paper/2604.11809/) (2026)
- [OrthoTrack: Continuous 6-DoF UAV Trajectory Estimation Anchored in Public Orthophotos](https://mlsanity.com/paper/2606.25245/) (2026)
- [No Labels, No Look-Ahead: Unsupervised Online Video Stabilization with Classical Priors](https://mlsanity.com/paper/2602.23141/) (2026)
- [Cross-View Feature Matching: Survey, Benchmarking, and Foundation-Model Perspectives](https://mlsanity.com/paper/2608.11093/) (2026)
- [RIPE++: Reinforced Keypoint Learning from Positive Pairs Only](https://mlsanity.com/paper/2608.19693/) (2026)
- [Unified and Efficient Point-Line Local Features](https://mlsanity.com/paper/2608.19894/) (2026)
- [Misanthrope: A Privacy-Preserving Keypoint Detector](https://mlsanity.com/paper/2608.23012/) (2026)
- [RoMa-Omega: What Feed-Forward 3D Models Know About Image Matching](https://mlsanity.com/paper/2609.09507/) (2026)
- [Towards Complementary Keypoint Detection via Mixture of Detectors](https://mlsanity.com/paper/2610.16762/) (2026)

## Abstract (arXiv)

Keypoints are what enable Structure-from-Motion (SfM) systems to scale to thousands of images. However, designing a keypoint detection objective is a non-trivial task, as SfM is non-differentiable. Typically, an auxiliary objective involving a descriptor is optimized. This however induces a dependency on the descriptor, which is undesirable. In this paper we propose a fully self-supervised and descriptor-free objective for keypoint detection, through reinforcement learning. To ensure training does not degenerate, we leverage a balanced top-K sampling strategy. While this already produces competitive models, we find that two qualitatively different types of detectors emerge, which are only able to detect light and dark keypoints respectively. To remedy this, we train a third detector, DaD, that optimizes the Kullback-Leibler divergence of the pointwise maximum of both light and dark detectors. Our approach significantly improve upon SotA across a range of benchmarks. Code and model weights are publicly available at https://github.com/parskatt/dad