# Improving the matching of deformable objects by learning to detect keypoints

arXiv: https://arxiv.org/abs/2309.00434 · HTML: https://mlsanity.com/paper/2309.00434/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 41 references that are in the map.
- First posted on arXiv 1 Sep 2023.

## Builds on (most cited first, 19 of 19)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [LoFTR: Detector-Free Local Feature Matching with Transformers](https://mlsanity.com/paper/2104.00680/) (2021)
- [D2-Net: A Trainable CNN for Joint Detection and Description of Local Features](https://mlsanity.com/paper/1905.03561/) (2019)
- [DISK: Learning local features with policy gradient](https://mlsanity.com/paper/2006.13566/) (2020)
- [LIFT: Learned Invariant Feature Transform](https://mlsanity.com/paper/1603.09114/) (2016)
- [Image Matching across Wide Baselines: From Paper to Practice](https://mlsanity.com/paper/2003.01587/) (2020)
- [LF-Net: Learning Local Features from Images](https://mlsanity.com/paper/1805.09662/) (2018)
- [ASLFeat: Learning Local Features of Accurate Shape and Localization](https://mlsanity.com/paper/2003.10071/) (2020)
- [Key.Net: Keypoint Detection by Handcrafted and Learned CNN Filters](https://mlsanity.com/paper/1904.00889/) (2019)
- [ContextDesc: Local Descriptor Augmentation with Cross-Modality Context](https://mlsanity.com/paper/1904.04084/) (2019)
- [ALIKE: Accurate and Lightweight Keypoint Detection and Descriptor Extraction](https://mlsanity.com/paper/2112.02906/) (2021)
- [TILDE: A Temporally Invariant Learned DEtector](https://mlsanity.com/paper/1411.4568/) (2014)
- [Quad-networks: unsupervised learning to rank for interest point detection](https://mlsanity.com/paper/1611.07571/) (2016)
- [GLAMpoints: Greedily Learned Accurate Match points](https://mlsanity.com/paper/1908.06812/) (2019)
- [SEKD: Self-Evolving Keypoint Detection and Description](https://mlsanity.com/paper/2006.05077/) (2020)
- [Extracting Deformation-Aware Local Features by Learning to Deform](https://mlsanity.com/paper/2111.10617/) (2021)
- [Learning Geodesic-Aware Local Features from RGB-D Images](https://mlsanity.com/paper/2203.12016/) (2022)
- [Learning to Detect Good Keypoints to Match Non-Rigid Objects in RGB Images](https://mlsanity.com/paper/2212.09589/) (2022)

## Built on by (most cited first, 1 of 1)

- [Deep Learning Reforms Image Matching: A Survey and Outlook](https://mlsanity.com/paper/2506.04619/) (2025)

## Abstract (arXiv)

We propose a novel learned keypoint detection method to increase the number of correct matches for the task of non-rigid image correspondence. By leveraging true correspondences acquired by matching annotated image pairs with a specified descriptor extractor, we train an end-to-end convolutional neural network (CNN) to find keypoint locations that are more appropriate to the considered descriptor. For that, we apply geometric and photometric warpings to images to generate a supervisory signal, allowing the optimization of the detector. Experiments demonstrate that our method enhances the Mean Matching Accuracy of numerous descriptors when used in conjunction with our detection method, while outperforming the state-of-the-art keypoint detectors on real images of non-rigid objects by 20 p.p. We also apply our method on the complex real-world task of object retrieval where our detector performs on par with the finest keypoint detectors currently available for this task. The source code and trained models are publicly available at https://github.com/verlab/LearningToDetect_PRL_2023