# Improving Worst Case Visual Localization Coverage via Place-specific Sub-selection in Multi-camera Systems

arXiv: https://arxiv.org/abs/2206.13883 · HTML: https://mlsanity.com/paper/2206.13883/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 62 references that are in the map.
- First posted on arXiv 28 Jun 2022.

## Builds on (most cited first, 30 of 33)

- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [NetVLAD: CNN architecture for weakly supervised place recognition](https://mlsanity.com/paper/1511.07247/) (2015)
- [From Coarse to Fine: Robust Hierarchical Localization at Large Scale](https://mlsanity.com/paper/1812.03506/) (2018)
- [DSAC - Differentiable RANSAC for Camera Localization](https://mlsanity.com/paper/1611.05705/) (2016)
- [Large-Scale Image Retrieval with Attentive Deep Local Features](https://mlsanity.com/paper/1612.06321/) (2016)
- [R2D2: Repeatable and Reliable Detector and Descriptor](https://mlsanity.com/paper/1906.06195/) (2019)
- [Pixel-Perfect Structure-from-Motion with Featuremetric Refinement](https://mlsanity.com/paper/2108.08291/) (2021)
- [Back to the Feature: Learning Robust Camera Localization from Pixels to Pose](https://mlsanity.com/paper/2103.09213/) (2021)
- [Learning with Average Precision: Training Image Retrieval with a Listwise Loss](https://mlsanity.com/paper/1906.07589/) (2019)
- [Neighbourhood Consensus Networks](https://mlsanity.com/paper/1810.10510/) (2018)
- [Hierarchical Scene Coordinate Classification and Regression for Visual Localization](https://mlsanity.com/paper/1909.06216/) (2019)
- [Deep Declarative Networks: A New Hope](https://mlsanity.com/paper/1909.04866/) (2019)
- [Robust Image Retrieval-based Visual Localization using Kapture](https://mlsanity.com/paper/2007.13867/) (2020)
- [VLocNet++: Deep Multitask Learning for Semantic Visual Localization and Odometry](https://mlsanity.com/paper/1804.08366/) (2018)
- [To Learn or Not to Learn: Visual Localization from Essential Matrices](https://mlsanity.com/paper/1908.01293/) (2019)
- [Semantics for Robotic Mapping, Perception and Interaction: A Survey](https://mlsanity.com/paper/2101.00443/) (2021)
- [Scalable Place Recognition Under Appearance Change for Autonomous Driving](https://mlsanity.com/paper/1908.00178/) (2019)
- [Benchmarking Image Retrieval for Visual Localization](https://mlsanity.com/paper/2011.11946/) (2020)
- [Ford Multi-AV Seasonal Dataset](https://mlsanity.com/paper/2003.07969/) (2020)
- [Solving the Blind Perspective-n-Point Problem End-To-End With Robust Differentiable Geometric Optimization](https://mlsanity.com/paper/2007.14628/) (2020)
- [GN-Net: The Gauss-Newton Loss for Multi-Weather Relocalization](https://mlsanity.com/paper/1904.11932/) (2019)
- [Robust and Precise Vehicle Localization based on Multi-sensor Fusion in Diverse City Scenes](https://mlsanity.com/paper/1711.05805/) (2017)
- [Is This The Right Place? Geometric-Semantic Pose Verification for Indoor Visual Localization](https://mlsanity.com/paper/1908.04598/) (2019)
- [Intelligent Reference Curation for Visual Place Recognition via Bayesian Selective Fusion](https://mlsanity.com/paper/2010.09228/) (2020)
- [Localization Requirements for Autonomous Vehicles](https://mlsanity.com/paper/1906.01061/) (2019)
- [A Hierarchical Dual Model of Environment- and Place-Specific Utility for Visual Place Recognition](https://mlsanity.com/paper/2107.02440/) (2021)
- [Efficient 2D-3D Matching for Multi-Camera Visual Localization](https://mlsanity.com/paper/1809.06445/) (2018)
- [Aerial Imagery based LIDAR Localization for Autonomous Vehicles](https://mlsanity.com/paper/2003.11192/) (2020)
- [Map Management for Efficient Long-Term Visual Localization in Outdoor Environments](https://mlsanity.com/paper/1808.02658/) (2018)

## Built on by (most cited first, 5 of 5)

- [Satellite Image Based Cross-view Localization for Autonomous Vehicle](https://mlsanity.com/paper/2207.13506/) (2022)
- [Stereo Visual Odometry with Deep Learning-Based Point and Line Feature Matching using an Attention Graph Neural Network](https://mlsanity.com/paper/2308.01125/) (2023)
- [Dynamically Modulating Visual Place Recognition Sequence Length For Minimum Acceptable Performance Scenarios](https://mlsanity.com/paper/2407.00863/) (2024)
- [Locking On: Leveraging Dynamic Vehicle-Imposed Motion Constraints to Improve Visual Localization](https://mlsanity.com/paper/2306.17529/) (2023)
- [DisPlace: Discriminative Place Projections for Multi-Reference Visual Place Recognition](https://mlsanity.com/paper/2605.30769/) (2026)

## Abstract (arXiv)

6-DoF visual localization systems utilize principled approaches rooted in 3D geometry to perform accurate camera pose estimation of images to a map. Current techniques use hierarchical pipelines and learned 2D feature extractors to improve scalability and increase performance. However, despite gains in typical recall@0.25m type metrics, these systems still have limited utility for real-world applications like autonomous vehicles because of their `worst' areas of performance - the locations where they provide insufficient recall at a certain required error tolerance. Here we investigate the utility of using `place specific configurations', where a map is segmented into a number of places, each with its own configuration for modulating the pose estimation step, in this case selecting a camera within a multi-camera system. On the Ford AV benchmark dataset, we demonstrate substantially improved worst-case localization performance compared to using off-the-shelf pipelines - minimizing the percentage of the dataset which has low recall at a certain error tolerance, as well as improved overall localization performance. Our proposed approach is particularly applicable to the crowdsharing model of autonomous vehicle deployment, where a fleet of AVs are regularly traversing a known route.