# ResMatch: Residual Attention Learning for Local Feature Matching

arXiv: https://arxiv.org/abs/2307.05180 · HTML: https://mlsanity.com/paper/2307.05180/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 55 references that are in the map.
- First posted on arXiv 11 Jul 2023.

## Builds on (most cited first, 30 of 34)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances](https://mlsanity.com/paper/1306.0895/) (2013)
- [Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention](https://mlsanity.com/paper/2006.16236/) (2020)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [LoFTR: Detector-Free Local Feature Matching with Transformers](https://mlsanity.com/paper/2104.00680/) (2021)
- [NetVLAD: CNN architecture for weakly supervised place recognition](https://mlsanity.com/paper/1511.07247/) (2015)
- [Efficient Transformers: A Survey](https://mlsanity.com/paper/2009.06732/) (2020)
- [From Coarse to Fine: Robust Hierarchical Localization at Large Scale](https://mlsanity.com/paper/1812.03506/) (2018)
- [Graph U-Nets](https://mlsanity.com/paper/1905.05178/) (2019)
- [D2-Net: A Trainable CNN for Joint Detection and Description of Local Features](https://mlsanity.com/paper/1905.03561/) (2019)
- [DISK: Learning local features with policy gradient](https://mlsanity.com/paper/2006.13566/) (2020)
- [YFCC100M: The New Data in Multimedia Research](https://mlsanity.com/paper/1503.01817/) (2015)
- [Working hard to know your neighbor's margins: Local descriptor learning loss](https://mlsanity.com/paper/1705.10872/) (2017)
- [Learning to Find Good Correspondences](https://mlsanity.com/paper/1711.05971/) (2017)
- [ASpanFormer: Detector-Free Image Matching with Adaptive Span Transformer](https://mlsanity.com/paper/2208.14201/) (2022)
- [MAGSAC++, a fast, reliable and accurate robust estimator](https://mlsanity.com/paper/1912.05909/) (2019)
- [Learning Two-View Correspondences and Geometry Using Order-Aware Network](https://mlsanity.com/paper/1908.04964/) (2019)
- [QuadTree Attention for Vision Transformers](https://mlsanity.com/paper/2201.02767/) (2022)
- [Learning to Match Features with Seeded Graph Matching Network](https://mlsanity.com/paper/2108.08771/) (2021)
- [RealFormer: Transformer Likes Residual Attention](https://mlsanity.com/paper/2012.11747/) (2020)
- [ClusterGNN: Cluster-based Coarse-to-Fine Graph Neural Network for Efficient Feature Matching](https://mlsanity.com/paper/2204.11700/) (2022)
- [Learnable Motion Coherence for Correspondence Pruning](https://mlsanity.com/paper/2011.14563/) (2020)
- [Matchable Image Retrieval by Learning from Surface Reconstruction](https://mlsanity.com/paper/1811.10343/) (2018)
- [An Evaluation of Feature Matchers for Fundamental Matrix Estimation](https://mlsanity.com/paper/1908.09474/) (2019)
- [DeepMatcher: A Deep Transformer-based Network for Robust and Accurate Local Feature Matching](https://mlsanity.com/paper/2301.02993/) (2023)

## Built on by (most cited first, 1 of 1)

- [Local Feature Matching Using Deep Learning: A Survey](https://mlsanity.com/paper/2401.17592/) (2024)

## Abstract (arXiv)

Attention-based graph neural networks have made great progress in feature matching learning. However, insight of how attention mechanism works for feature matching is lacked in the literature. In this paper, we rethink cross- and self-attention from the viewpoint of traditional feature matching and filtering. In order to facilitate the learning of matching and filtering, we inject the similarity of descriptors and relative positions into cross- and self-attention score, respectively. In this way, the attention can focus on learning residual matching and filtering functions with reference to the basic functions of measuring visual and spatial correlation. Moreover, we mine intra- and inter-neighbors according to the similarity of descriptors and relative positions. Then sparse attention for each point can be performed only within its neighborhoods to acquire higher computation efficiency. Feature matching networks equipped with our full and sparse residual attention learning strategies are termed ResMatch and sResMatch respectively. Extensive experiments, including feature matching, pose estimation and visual localization, confirm the superiority of our networks.