# Learning to Detect Good Keypoints to Match Non-Rigid Objects in RGB Images

arXiv: https://arxiv.org/abs/2212.09589 · HTML: https://mlsanity.com/paper/2212.09589/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 33 references that are in the map.
- First posted on arXiv 13 Dec 2022.

## Builds on (most cited first, 13 of 13)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [D2-Net: A Trainable CNN for Joint Detection and Description of Local Features](https://mlsanity.com/paper/1905.03561/) (2019)
- [DISK: Learning local features with policy gradient](https://mlsanity.com/paper/2006.13566/) (2020)
- [Image Matching across Wide Baselines: From Paper to Practice](https://mlsanity.com/paper/2003.01587/) (2020)
- [ASLFeat: Learning Local Features of Accurate Shape and Localization](https://mlsanity.com/paper/2003.10071/) (2020)
- [ContextDesc: Local Descriptor Augmentation with Cross-Modality Context](https://mlsanity.com/paper/1904.04084/) (2019)
- [Quad-networks: unsupervised learning to rank for interest point detection](https://mlsanity.com/paper/1611.07571/) (2016)
- [GLAMpoints: Greedily Learned Accurate Match points](https://mlsanity.com/paper/1908.06812/) (2019)
- [SEKD: Self-Evolving Keypoint Detection and Description](https://mlsanity.com/paper/2006.05077/) (2020)
- [Extracting Deformation-Aware Local Features by Learning to Deform](https://mlsanity.com/paper/2111.10617/) (2021)
- [Learning Geodesic-Aware Local Features from RGB-D Images](https://mlsanity.com/paper/2203.12016/) (2022)

## Built on by (most cited first, 2 of 2)

- [Enhancing Deformable Local Features by Jointly Learning to Detect and Describe Keypoints](https://mlsanity.com/paper/2304.00583/) (2023)
- [Improving the matching of deformable objects by learning to detect keypoints](https://mlsanity.com/paper/2309.00434/) (2023)

## Abstract (arXiv)

We present a novel learned keypoint detection method designed to maximize the number of correct matches for the task of non-rigid image correspondence. Our training framework uses true correspondences, obtained by matching annotated image pairs with a predefined descriptor extractor, as a ground-truth to train a convolutional neural network (CNN). We optimize the model architecture by applying known geometric transformations to images as the supervisory signal. Experiments show that our method outperforms the state-of-the-art keypoint detector on real images of non-rigid objects by 20 p.p. on Mean Matching Accuracy and also improves the matching performance of several descriptors when coupled with our detection method. We also employ the proposed method in one challenging realworld application: object retrieval, where our detector exhibits performance on par with the best available keypoint detectors. The source code and trained model are publicly available at https://github.com/verlab/LearningToDetect SIBGRAPI 2022