# Retrieval and Localization with Observation Constraints

arXiv: https://arxiv.org/abs/2108.08516 · HTML: https://mlsanity.com/paper/2108.08516/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 65 references that are in the map.
- First posted on arXiv 19 Aug 2021.

## Builds on (most cited first, 30 of 32)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [Dual Attention Network for Scene Segmentation](https://mlsanity.com/paper/1809.02983/) (2018)
- [NetVLAD: CNN architecture for weakly supervised place recognition](https://mlsanity.com/paper/1511.07247/) (2015)
- [BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentation](https://mlsanity.com/paper/1808.00897/) (2018)
- [From Coarse to Fine: Robust Hierarchical Localization at Large Scale](https://mlsanity.com/paper/1812.03506/) (2018)
- [Fine-tuning CNN Image Retrieval with No Human Annotation](https://mlsanity.com/paper/1711.02512/) (2017)
- [D2-Net: A Trainable CNN for Joint Detection and Description of Local Features](https://mlsanity.com/paper/1905.03561/) (2019)
- [Google Landmarks Dataset v2 -- A Large-Scale Benchmark for Instance-Level Recognition and Retrieval](https://mlsanity.com/paper/2004.01804/) (2020)
- [LIFT: Learned Invariant Feature Transform](https://mlsanity.com/paper/1603.09114/) (2016)
- [BiSeNet V2: Bilateral Network with Guided Aggregation for Real-time Semantic Segmentation](https://mlsanity.com/paper/2004.02147/) (2020)
- [Large-Scale Image Retrieval with Attentive Deep Local Features](https://mlsanity.com/paper/1612.06321/) (2016)
- [Large Kernel Matters -- Improve Semantic Segmentation by Global Convolutional Network](https://mlsanity.com/paper/1703.02719/) (2017)
- [R2D2: Repeatable and Reliable Detector and Descriptor](https://mlsanity.com/paper/1906.06195/) (2019)
- [Revisiting Oxford and Paris: Large-Scale Image Retrieval Benchmarking](https://mlsanity.com/paper/1803.11285/) (2018)
- [Learning with Average Precision: Training Image Retrieval with a Listwise Loss](https://mlsanity.com/paper/1906.07589/) (2019)
- [Neighbourhood Consensus Networks](https://mlsanity.com/paper/1810.10510/) (2018)
- [End-to-end Learning of Deep Visual Representations for Image Retrieval](https://mlsanity.com/paper/1610.07940/) (2016)
- [In Defense of Pre-trained ImageNet Architectures for Real-time Semantic Segmentation of Road-driving Images](https://mlsanity.com/paper/1903.08469/) (2019)
- [HarDNet: A Low Memory Traffic Network](https://mlsanity.com/paper/1909.00948/) (2019)
- [Robust Image Retrieval-based Visual Localization using Kapture](https://mlsanity.com/paper/2007.13867/) (2020)
- [Aggregating Deep Convolutional Features for Image Retrieval](https://mlsanity.com/paper/1510.07493/) (2015)
- [Interlaced Sparse Self-Attention for Semantic Segmentation](https://mlsanity.com/paper/1907.12273/) (2019)
- [Online Invariance Selection for Local Feature Descriptors](https://mlsanity.com/paper/2007.08988/) (2020)

## Built on by (most cited first, 6 of 6)

- [Local Feature Matching Using Deep Learning: A Survey](https://mlsanity.com/paper/2401.17592/) (2024)
- [Pose Refinement with Joint Optimization of Visual Points and Lines](https://mlsanity.com/paper/2110.03940/) (2021)
- [UFM: Unified Feature Matching Pre-training with Multi-Modal Image Assistants](https://mlsanity.com/paper/2503.21820/) (2025)
- [A Real-Time Fusion Framework for Long-term Visual Localization](https://mlsanity.com/paper/2210.09757/) (2022)
- [SGL: Structure Guidance Learning for Camera Localization](https://mlsanity.com/paper/2304.05571/) (2023)
- [Multiview Image-Based Localization](https://mlsanity.com/paper/2503.23577/) (2025)

## Abstract (arXiv)

Accurate visual re-localization is very critical to many artificial intelligence applications, such as augmented reality, virtual reality, robotics and autonomous driving. To accomplish this task, we propose an integrated visual re-localization method called RLOCS by combining image retrieval, semantic consistency and geometry verification to achieve accurate estimations. The localization pipeline is designed as a coarse-to-fine paradigm. In the retrieval part, we cascade the architecture of ResNet101-GeM-ArcFace and employ DBSCAN followed by spatial verification to obtain a better initial coarse pose. We design a module called observation constraints, which combines geometry information and semantic consistency for filtering outliers. Comprehensive experiments are conducted on open datasets, including retrieval on R-Oxford5k and R-Paris6k, semantic segmentation on Cityscapes, localization on Aachen Day-Night and InLoc. By creatively modifying separate modules in the total pipeline, our method achieves many performance improvements on the challenging localization benchmarks.