# Enhancing Deformable Local Features by Jointly Learning to Detect and Describe Keypoints

arXiv: https://arxiv.org/abs/2304.00583 · HTML: https://mlsanity.com/paper/2304.00583/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 47 references that are in the map.
- First posted on arXiv 2 Apr 2023.

## Builds on (most cited first, 24 of 24)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [D2-Net: A Trainable CNN for Joint Detection and Description of Local Features](https://mlsanity.com/paper/1905.03561/) (2019)
- [DISK: Learning local features with policy gradient](https://mlsanity.com/paper/2006.13566/) (2020)
- [LIFT: Learned Invariant Feature Transform](https://mlsanity.com/paper/1603.09114/) (2016)
- [HPatches: A benchmark and evaluation of handcrafted and learned local descriptors](https://mlsanity.com/paper/1704.05939/) (2017)
- [Large-Scale Image Retrieval with Attentive Deep Local Features](https://mlsanity.com/paper/1612.06321/) (2016)
- [Image Matching across Wide Baselines: From Paper to Practice](https://mlsanity.com/paper/2003.01587/) (2020)
- [Working hard to know your neighbor's margins: Local descriptor learning loss](https://mlsanity.com/paper/1705.10872/) (2017)
- [LF-Net: Learning Local Features from Images](https://mlsanity.com/paper/1805.09662/) (2018)
- [SOSNet: Second Order Similarity Regularization for Local Descriptor Learning](https://mlsanity.com/paper/1904.05019/) (2019)
- [ASLFeat: Learning Local Features of Accurate Shape and Localization](https://mlsanity.com/paper/2003.10071/) (2020)
- [Key.Net: Keypoint Detection by Handcrafted and Learned CNN Filters](https://mlsanity.com/paper/1904.00889/) (2019)
- [DeepDeform: Learning Non-rigid RGB-D Reconstruction with Semi-supervised Data](https://mlsanity.com/paper/1912.04302/) (2019)
- [VolumeDeform: Real-time Volumetric Non-rigid Reconstruction](https://mlsanity.com/paper/1603.08161/) (2016)
- [GeoDesc: Learning Local Descriptors by Integrating Geometry Constraints](https://mlsanity.com/paper/1807.06294/) (2018)
- [Detect-to-Retrieve: Efficient Regional Aggregation for Image Search](https://mlsanity.com/paper/1812.01584/) (2018)
- [Beyond Cartesian Representations for Local Descriptors](https://mlsanity.com/paper/1908.05547/) (2019)
- [AdaLAM: Revisiting Handcrafted Outlier Detection](https://mlsanity.com/paper/2006.04250/) (2020)
- [Extracting Deformation-Aware Local Features by Learning to Deform](https://mlsanity.com/paper/2111.10617/) (2021)
- [Learning Geodesic-Aware Local Features from RGB-D Images](https://mlsanity.com/paper/2203.12016/) (2022)
- [Learning to Detect Good Keypoints to Match Non-Rigid Objects in RGB Images](https://mlsanity.com/paper/2212.09589/) (2022)

## Built on by (most cited first, 11 of 11)

- [OmniGlue: Generalizable Feature Matching with Foundation Model Guidance](https://mlsanity.com/paper/2405.12979/) (2024)
- [Tracking and Mapping in Medical Computer Vision: A Review](https://mlsanity.com/paper/2310.11475/) (2023)
- [RIPE: Reinforcement Learning on Unlabeled Image Pairs for Robust Keypoint Extraction](https://mlsanity.com/paper/2507.04839/) (2025)
- [Deep Learning Reforms Image Matching: A Survey and Outlook](https://mlsanity.com/paper/2506.04619/) (2025)
- [Progressive Retinal Image Registration via Global and Local Deformable Transformations](https://mlsanity.com/paper/2409.01068/) (2024)
- [RIDE: Self-Supervised Learning of Rotation-Equivariant Keypoint Detection and Invariant Description for Endoscopy](https://mlsanity.com/paper/2309.09563/) (2023)
- [Leveraging Semantic Cues from Foundation Vision Models for Enhanced Local Feature Correspondence](https://mlsanity.com/paper/2410.09533/) (2024)
- [Textureless Deformable Surface Reconstruction with Invisible Markers](https://mlsanity.com/paper/2308.13678/) (2023)
- [EndoMatcher: Generalizable Endoscopic Image Matcher via Multi-Domain Pre-training for Robot-Assisted Surgery](https://mlsanity.com/paper/2508.05205/) (2025)
- [IUP-Pose: Decoupled Iterative Uncertainty Propagation for Real-time Relative Pose Regression via Implicit Dense Alignment v1](https://mlsanity.com/paper/2603.19625/) (2026)
- [RIPE++: Reinforced Keypoint Learning from Positive Pairs Only](https://mlsanity.com/paper/2608.19693/) (2026)

## Abstract (arXiv)

Local feature extraction is a standard approach in computer vision for tackling important tasks such as image matching and retrieval. The core assumption of most methods is that images undergo affine transformations, disregarding more complicated effects such as non-rigid deformations. Furthermore, incipient works tailored for non-rigid correspondence still rely on keypoint detectors designed for rigid transformations, hindering performance due to the limitations of the detector. We propose DALF (Deformation-Aware Local Features), a novel deformation-aware network for jointly detecting and describing keypoints, to handle the challenging problem of matching deformable surfaces. All network components work cooperatively through a feature fusion approach that enforces the descriptors' distinctiveness and invariance. Experiments using real deforming objects showcase the superiority of our method, where it delivers 8% improvement in matching scores compared to the previous best results. Our approach also enhances the performance of two real-world applications: deformable object retrieval and non-rigid 3D surface registration. Code for training, inference, and applications are publicly available at https://verlab.dcc.ufmg.br/descriptors/dalf_cvpr23.