# PlaceFormer: Transformer-based Visual Place Recognition using Multi-Scale Patch Selection and Fusion

arXiv: https://arxiv.org/abs/2401.13082 · HTML: https://mlsanity.com/paper/2401.13082/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 35 references that are in the map.
- First posted on arXiv 23 Jan 2024.

## Builds on (most cited first, 18 of 18)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [NetVLAD: CNN architecture for weakly supervised place recognition](https://mlsanity.com/paper/1511.07247/) (2015)
- [Fine-tuning CNN Image Retrieval with No Human Annotation](https://mlsanity.com/paper/1711.02512/) (2017)
- [D2-Net: A Trainable CNN for Joint Detection and Description of Local Features](https://mlsanity.com/paper/1905.03561/) (2019)
- [Patch-NetVLAD: Multi-Scale Fusion of Locally-Global Descriptors for Place Recognition](https://mlsanity.com/paper/2103.01486/) (2021)
- [Rethinking Visual Geo-localization for Large-Scale Applications](https://mlsanity.com/paper/2204.02287/) (2022)
- [MixVPR: Feature Mixing for Visual Place Recognition](https://mlsanity.com/paper/2303.02190/) (2023)
- [End-to-end Learning of Deep Visual Representations for Image Retrieval](https://mlsanity.com/paper/1610.07940/) (2016)
- [Self-supervising Fine-grained Region Similarities for Large-scale Image Localization](https://mlsanity.com/paper/2006.03926/) (2020)
- [TransVPR: Transformer-based place recognition with multi-level attention aggregation](https://mlsanity.com/paper/2201.02001/) (2022)
- [R2Former: Unified Retrieval and Reranking Transformer for Place Recognition](https://mlsanity.com/paper/2304.03410/) (2023)
- [Deep Learning Features at Scale for Visual Place Recognition](https://mlsanity.com/paper/1701.05105/) (2017)
- [Training Vision Transformers for Image Retrieval](https://mlsanity.com/paper/2102.05644/) (2021)
- [Deep Visual Geo-localization Benchmark](https://mlsanity.com/paper/2204.03444/) (2022)
- [A Holistic Visual Place Recognition Approach using Lightweight CNNs for Significant ViewPoint and Appearance Changes](https://mlsanity.com/paper/1811.03032/) (2018)
- [Detect-to-Retrieve: Efficient Regional Aggregation for Image Search](https://mlsanity.com/paper/1812.01584/) (2018)

## Built on by (most cited first, 4 of 4)

- [ZeroSCD: Zero-Shot Street Scene Change Detection](https://mlsanity.com/paper/2409.15255/) (2024)
- [RING#: PR-by-PE Global Localization with Roto-translation Equivariant Gram Learning](https://mlsanity.com/paper/2409.00206/) (2024)
- [CLOVER: Context-aware Long-term Object Viewpoint- and Environment- Invariant Representation Learning](https://mlsanity.com/paper/2407.09718/) (2024)
- [Revisiting Change Detection Methods for their Application to Serac Fall Time-Lapse Monitoring](https://mlsanity.com/paper/2605.28100/) (2026)

## Abstract (arXiv)

Visual place recognition is a challenging task in the field of computer vision, and autonomous robotics and vehicles, which aims to identify a location or a place from visual inputs. Contemporary methods in visual place recognition employ convolutional neural networks and utilize every region within the image for the place recognition task. However, the presence of dynamic and distracting elements in the image may impact the effectiveness of the place recognition process. Therefore, it is meaningful to focus on task-relevant regions of the image for improved recognition. In this paper, we present PlaceFormer, a novel transformer-based approach for visual place recognition. PlaceFormer employs patch tokens from the transformer to create global image descriptors, which are then used for image retrieval. To re-rank the retrieved images, PlaceFormer merges the patch tokens from the transformer to form multi-scale patches. Utilizing the transformer's self-attention mechanism, it selects patches that correspond to task-relevant areas in an image. These selected patches undergo geometric verification, generating similarity scores across different patch sizes. Subsequently, spatial scores from each patch size are fused to produce a final similarity score. This score is then used to re-rank the images initially retrieved using global image descriptors. Extensive experiments on benchmark datasets demonstrate that PlaceFormer outperforms several state-of-the-art methods in terms of accuracy and computational efficiency, requiring less time and memory.