# Semantic Pose Verification for Outdoor Visual Localization with Self-supervised Contrastive Learning

arXiv: https://arxiv.org/abs/2203.16945 · HTML: https://mlsanity.com/paper/2203.16945/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 53 references that are in the map.
- First posted on arXiv 31 Mar 2022.

## Builds on (most cited first, 27 of 27)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [NetVLAD: CNN architecture for weakly supervised place recognition](https://mlsanity.com/paper/1511.07247/) (2015)
- [What Makes for Good Views for Contrastive Learning?](https://mlsanity.com/paper/2005.10243/) (2020)
- [Fine-tuning CNN Image Retrieval with No Human Annotation](https://mlsanity.com/paper/1711.02512/) (2017)
- [Keystroke Dynamics as Part of Lifelogging](https://mlsanity.com/paper/2010.10144/) (2020)
- [LIFT: Learned Invariant Feature Transform](https://mlsanity.com/paper/1603.09114/) (2016)
- [High-Resolution Representations for Labeling Pixels and Regions](https://mlsanity.com/paper/1904.04514/) (2019)
- [Revisiting Oxford and Paris: Large-Scale Image Retrieval Benchmarking](https://mlsanity.com/paper/1803.11285/) (2018)
- [Particular object retrieval with integral max-pooling of CNN activations](https://mlsanity.com/paper/1511.05879/) (2015)
- [Convolutional neural network architecture for geometric matching](https://mlsanity.com/paper/1703.05593/) (2017)
- [Self-supervising Fine-grained Region Similarities for Large-scale Image Localization](https://mlsanity.com/paper/2006.03926/) (2020)
- [Deep Learning Features at Scale for Visual Place Recognition](https://mlsanity.com/paper/1701.05105/) (2017)
- [Visual Instance Retrieval with Deep Convolutional Networks](https://mlsanity.com/paper/1412.6574/) (2014)
- [Large-scale, real-time visual-inertial localization revisited](https://mlsanity.com/paper/1907.00338/) (2019)
- [CASENet: Deep Category-Aware Semantic Edge Detection](https://mlsanity.com/paper/1705.09759/) (2017)
- [Is This The Right Place? Geometric-Semantic Pose Verification for Indoor Visual Localization](https://mlsanity.com/paper/1908.04598/) (2019)
- [Long-term Visual Localization using Semantically Segmented Images](https://mlsanity.com/paper/1801.05269/) (2018)
- [Factors of Transferability for a Generic ConvNet Representation](https://mlsanity.com/paper/1406.5774/) (2014)
- [VLASE: Vehicle Localization by Aggregating Semantic Edges](https://mlsanity.com/paper/1807.02536/) (2018)
- [Panorama to panorama matching for location recognition](https://mlsanity.com/paper/1704.06591/) (2017)
- [Training Semantic Descriptors for Image-Based Localization](https://mlsanity.com/paper/2202.01212/) (2022)

## Built on by (most cited first, 5 of 5)

- [Review on Panoramic Imaging and Its Applications in Scene Understanding](https://mlsanity.com/paper/2205.05570/) (2022)
- [GV-Bench: Benchmarking Local Feature Matching for Geometric Verification of Long-term Loop Closure Detection](https://mlsanity.com/paper/2407.11736/) (2024)
- [SPVLoc: Semantic Panoramic Viewport Matching for 6D Camera Localization in Unseen Environments](https://mlsanity.com/paper/2404.10527/) (2024)
- [Single-Shot Global Localization via Graph-Theoretic Correspondence Matching](https://mlsanity.com/paper/2306.03641/) (2023)
- [VISTA: Monocular Segmentation-Based Mapping for Appearance and View-Invariant Global Localization](https://mlsanity.com/paper/2507.11653/) (2025)

## Abstract (arXiv)

Any city-scale visual localization system has to overcome long-term appearance changes, such as varying illumination conditions or seasonal changes between query and database images. Since semantic content is more robust to such changes, we exploit semantic information to improve visual localization. In our scenario, the database consists of gnomonic views generated from panoramic images (e.g. Google Street View) and query images are collected with a standard field-of-view camera at a different time. To improve localization, we check the semantic similarity between query and database images, which is not trivial since the position and viewpoint of the cameras do not exactly match. To learn similarity, we propose training a CNN in a self-supervised fashion with contrastive learning on a dataset of semantically segmented images. With experiments we showed that this semantic similarity estimation approach works better than measuring the similarity at pixel-level. Finally, we used the semantic similarity scores to verify the retrievals obtained by a state-of-the-art visual localization method and observed that contrastive learning-based pose verification increases top-1 recall value to 0.90 which corresponds to a 2% improvement.