# S-TREK: Sequential Translation and Rotation Equivariant Keypoints for local feature extraction

arXiv: https://arxiv.org/abs/2308.14598 · HTML: https://mlsanity.com/paper/2308.14598/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 48 references that are in the map.
- First posted on arXiv 28 Aug 2023.

## Builds on (most cited first, 26 of 26)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [Group Equivariant Convolutional Networks](https://mlsanity.com/paper/1602.07576/) (2016)
- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [LoFTR: Detector-Free Local Feature Matching with Transformers](https://mlsanity.com/paper/2104.00680/) (2021)
- [MegaDepth: Learning Single-View Depth Prediction from Internet Photos](https://mlsanity.com/paper/1804.00607/) (2018)
- [D2-Net: A Trainable CNN for Joint Detection and Description of Local Features](https://mlsanity.com/paper/1905.03561/) (2019)
- [DISK: Learning local features with policy gradient](https://mlsanity.com/paper/2006.13566/) (2020)
- [LIFT: Learned Invariant Feature Transform](https://mlsanity.com/paper/1603.09114/) (2016)
- [HPatches: A benchmark and evaluation of handcrafted and learned local descriptors](https://mlsanity.com/paper/1704.05939/) (2017)
- [Image Matching across Wide Baselines: From Paper to Practice](https://mlsanity.com/paper/2003.01587/) (2020)
- [Working hard to know your neighbor's margins: Local descriptor learning loss](https://mlsanity.com/paper/1705.10872/) (2017)
- [R2D2: Repeatable and Reliable Detector and Descriptor](https://mlsanity.com/paper/1906.06195/) (2019)
- [LF-Net: Learning Local Features from Images](https://mlsanity.com/paper/1805.09662/) (2018)
- [SOSNet: Second Order Similarity Regularization for Local Descriptor Learning](https://mlsanity.com/paper/1904.05019/) (2019)
- [ASLFeat: Learning Local Features of Accurate Shape and Localization](https://mlsanity.com/paper/2003.10071/) (2020)
- [Key.Net: Keypoint Detection by Handcrafted and Learned CNN Filters](https://mlsanity.com/paper/1904.00889/) (2019)
- [TILDE: A Temporally Invariant Learned DEtector](https://mlsanity.com/paper/1411.4568/) (2014)
- [Reinforced Feature Points: Optimizing Feature Detection and Description for a High-Level Task](https://mlsanity.com/paper/1912.00623/) (2019)
- [Quad-networks: unsupervised learning to rank for interest point detection](https://mlsanity.com/paper/1611.07571/) (2016)
- [Beyond Cartesian Representations for Local Descriptors](https://mlsanity.com/paper/1908.05547/) (2019)
- [Self-Supervised Equivariant Learning for Oriented Keypoint Detection](https://mlsanity.com/paper/2204.08613/) (2022)
- [MD-Net: Multi-Detector for Local Feature Extraction](https://mlsanity.com/paper/2208.05350/) (2022)

## Built on by (most cited first, 14 of 14)

- [Local Feature Matching Using Deep Learning: A Survey](https://mlsanity.com/paper/2401.17592/) (2024)
- [Matching 2D Images in 3D: Metric Relative Pose from Metric Correspondences](https://mlsanity.com/paper/2404.06337/) (2024)
- [Steerers: A framework for rotation equivariant keypoint descriptors](https://mlsanity.com/paper/2312.02152/) (2023)
- [DaD: Distilled Reinforcement Learning for Diverse Keypoint Detection](https://mlsanity.com/paper/2503.07347/) (2025)
- [Deep Learning Reforms Image Matching: A Survey and Outlook](https://mlsanity.com/paper/2506.04619/) (2025)
- [Affine steerers for structured keypoint description](https://mlsanity.com/paper/2408.14186/) (2024)
- [GMM-IKRS: Gaussian Mixture Models for Interpretable Keypoint Refinement and Scoring](https://mlsanity.com/paper/2408.17149/) (2024)
- [Image Matching Filtering and Refinement by Planes and Beyond](https://mlsanity.com/paper/2411.09484/) (2024)
- [RaCo: Ranking and Covariance for Practical Learned Keypoints](https://mlsanity.com/paper/2602.15755/) (2026)
- [A Streamlined Attention-Based Network for Descriptor Extraction](https://mlsanity.com/paper/2601.13126/) (2026)
- [TerraSky3D: Multi-View Reconstructions of European Landmarks in 4K](https://mlsanity.com/paper/2603.28287/) (2026)
- [From Pairs to Sequences: Track-Aware Policy Gradients for Keypoint Detection](https://mlsanity.com/paper/2602.20630/) (2026)
- [RIPE++: Reinforced Keypoint Learning from Positive Pairs Only](https://mlsanity.com/paper/2608.19693/) (2026)
- [On Making SE(2)-Invariant Networks Optimal](https://mlsanity.com/paper/2610.17424/) (2026)

## Abstract (arXiv)

In this work we introduce S-TREK, a novel local feature extractor that combines a deep keypoint detector, which is both translation and rotation equivariant by design, with a lightweight deep descriptor extractor. We train the S-TREK keypoint detector within a framework inspired by reinforcement learning, where we leverage a sequential procedure to maximize a reward directly related to keypoint repeatability. Our descriptor network is trained following a "detect, then describe" approach, where the descriptor loss is evaluated only at those locations where keypoints have been selected by the already trained detector. Extensive experiments on multiple benchmarks confirm the effectiveness of our proposed method, with S-TREK often outperforming other state-of-the-art methods in terms of repeatability and quality of the recovered poses, especially when dealing with in-plane rotations.