# ReF -- Rotation Equivariant Features for Local Feature Matching

arXiv: https://arxiv.org/abs/2203.05206 · HTML: https://mlsanity.com/paper/2203.05206/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 33 references that are in the map.
- First posted on arXiv 10 Mar 2022.

## Builds on (most cited first, 20 of 20)

- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles](https://mlsanity.com/paper/1705.05065/) (2017)
- [General E(2)-Equivariant Steerable CNNs](https://mlsanity.com/paper/1911.08251/) (2019)
- [Steerable CNNs](https://mlsanity.com/paper/1612.08498/) (2016)
- [Patch-NetVLAD: Multi-Scale Fusion of Locally-Global Descriptors for Place Recognition](https://mlsanity.com/paper/2103.01486/) (2021)
- [R2D2: Repeatable and Reliable Detector and Descriptor](https://mlsanity.com/paper/1906.06195/) (2019)
- [ReDet: A Rotation-equivariant Detector for Aerial Object Detection](https://mlsanity.com/paper/2103.07733/) (2021)
- [TransVPR: Transformer-based place recognition with multi-level attention aggregation](https://mlsanity.com/paper/2201.02001/) (2022)
- [VPR-Bench: An Open-Source Visual Place Recognition Evaluation Framework with Quantifiable Viewpoint and Appearance Change](https://mlsanity.com/paper/2005.08135/) (2020)
- [Where is your place, Visual Place Recognition?](https://mlsanity.com/paper/2103.06443/) (2021)
- [Local Descriptors Optimized for Average Precision](https://mlsanity.com/paper/1804.05312/) (2018)
- [LoST? Appearance-Invariant Place Recognition for Opposite Viewpoints using Visual Semantics](https://mlsanity.com/paper/1804.05526/) (2018)
- [Single-Perspective Warps in Natural Image Stitching](https://mlsanity.com/paper/1802.04645/) (2018)
- [Group Equivariant Deep Reinforcement Learning](https://mlsanity.com/paper/2007.03437/) (2020)
- [Steerable Partial Differential Operators for Equivariant Neural Networks](https://mlsanity.com/paper/2106.10163/) (2021)
- [Rotation-Equivariant Deep Learning for Diffusion MRI](https://mlsanity.com/paper/2102.06942/) (2021)
- [RoRD: Rotation-Robust Descriptors and Orthographic Views for Local Feature Matching](https://mlsanity.com/paper/2103.08573/) (2021)
- [A Hierarchical Dual Model of Environment- and Place-Specific Utility for Visual Place Recognition](https://mlsanity.com/paper/2107.02440/) (2021)
- [Danish Airs and Grounds: A Dataset for Aerial-to-Street-Level Place Recognition and Localization](https://mlsanity.com/paper/2202.01821/) (2022)

## Built on by (most cited first, 2 of 2)

- [DRKF: Distilled Rotated Kernel Fusion for Efficient Rotation Invariant Descriptors in Local Feature Matching](https://mlsanity.com/paper/2209.10907/) (2022)
- [SIM2E: Benchmarking the Group Equivariant Capability of Correspondence Matching Algorithms](https://mlsanity.com/paper/2208.09896/) (2022)

## Abstract (arXiv)

Sparse local feature matching is pivotal for many computer vision and robotics tasks. To improve their invariance to challenging appearance conditions and viewing angles, and hence their usefulness, existing learning-based methods have primarily focused on data augmentation-based training. In this work, we propose an alternative, complementary approach that centers on inducing bias in the model architecture itself to generate `rotation-specific' features using Steerable E2-CNNs, that are then group-pooled to achieve rotation-invariant local features. We demonstrate that this high performance, rotation-specific coverage from the steerable CNNs can be expanded to all rotation angles by combining it with augmentation-trained standard CNNs which have broader coverage but are often inaccurate, thus creating a state-of-the-art rotation-robust local feature matcher. We benchmark our proposed methods against existing techniques on HPatches and a newly proposed UrbanScenes3D-Air dataset for visual place recognition. Furthermore, we present a detailed analysis of the performance effects of ensembling, robust estimation, network architecture variations, and the use of rotation priors.