# Visual Localization using Imperfect 3D Models from the Internet

arXiv: https://arxiv.org/abs/2304.05947 · HTML: https://mlsanity.com/paper/2304.05947/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 52 of its 117 references that are in the map.
- First posted on arXiv 12 Apr 2023.

## Builds on (most cited first, 30 of 52)

- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [LoFTR: Detector-Free Local Feature Matching with Transformers](https://mlsanity.com/paper/2104.00680/) (2021)
- [NetVLAD: CNN architecture for weakly supervised place recognition](https://mlsanity.com/paper/1511.07247/) (2015)
- [From Coarse to Fine: Robust Hierarchical Localization at Large Scale](https://mlsanity.com/paper/1812.03506/) (2018)
- [CosyPose: Consistent multi-view multi-object 6D pose estimation](https://mlsanity.com/paper/2008.08465/) (2020)
- [DSAC - Differentiable RANSAC for Camera Localization](https://mlsanity.com/paper/1611.05705/) (2016)
- [YFCC100M: The New Data in Multimedia Research](https://mlsanity.com/paper/1503.01817/) (2015)
- [Image Matching across Wide Baselines: From Paper to Practice](https://mlsanity.com/paper/2003.01587/) (2020)
- [Geometric Loss Functions for Camera Pose Regression with Deep Learning](https://mlsanity.com/paper/1704.00390/) (2017)
- [Back to the Feature: Learning Robust Camera Localization from Pixels to Pose](https://mlsanity.com/paper/2103.09213/) (2021)
- [Wide-Area Image Geolocalization with Aerial Reference Imagery](https://mlsanity.com/paper/1510.03743/) (2015)
- [Learning with Average Precision: Training Image Retrieval with a Listwise Loss](https://mlsanity.com/paper/1906.07589/) (2019)
- [Understanding the Limitations of CNN-based Absolute Camera Pose Regression](https://mlsanity.com/paper/1903.07504/) (2019)
- [Visual Camera Re-Localization from RGB and RGB-D Images Using DSAC](https://mlsanity.com/paper/2002.12324/) (2020)
- [MAGSAC++, a fast, reliable and accurate robust estimator](https://mlsanity.com/paper/1912.05909/) (2019)
- [EPOS: Estimating 6D Pose of Objects with Symmetries](https://mlsanity.com/paper/2004.00605/) (2020)
- [Learning Less is More - 6D Camera Localization via 3D Surface Regression](https://mlsanity.com/paper/1711.10228/) (2017)
- [Image-based localization using LSTMs for structured feature correlation](https://mlsanity.com/paper/1611.07890/) (2016)
- [Graph-Cut RANSAC](https://mlsanity.com/paper/1706.00984/) (2017)
- [Patch2Pix: Epipolar-Guided Pixel-Level Correspondences](https://mlsanity.com/paper/2012.01909/) (2020)
- [MAGSAC: marginalizing sample consensus](https://mlsanity.com/paper/1803.07469/) (2018)
- [End-to-end Learning of Deep Visual Representations for Image Retrieval](https://mlsanity.com/paper/1610.07940/) (2016)
- [LENS: Localization enhanced by NeRF synthesis](https://mlsanity.com/paper/2110.06558/) (2021)
- [Learning Multi-Scene Absolute Pose Regression with Transformers](https://mlsanity.com/paper/2103.11468/) (2021)
- [Camera Relocalization by Computing Pairwise Relative Poses Using Convolutional Neural Network](https://mlsanity.com/paper/1707.09733/) (2017)
- [ROCA: Robust CAD Model Retrieval and Alignment from a Single Image](https://mlsanity.com/paper/2112.01988/) (2021)
- [Robust Image Retrieval-based Visual Localization using Kapture](https://mlsanity.com/paper/2007.13867/) (2020)
- [On the Limits of Pseudo Ground Truth in Visual Camera Re-localisation](https://mlsanity.com/paper/2109.00524/) (2021)
- [Learning to Navigate the Energy Landscape](https://mlsanity.com/paper/1603.05772/) (2016)

## Built on by (most cited first, 20 of 20)

- [AerialMegaDepth: Learning Aerial-Ground Reconstruction and View Synthesis](https://mlsanity.com/paper/2504.13157/) (2025)
- [Close, But Not There: Boosting Geographic Distance Sensitivity in Visual Place Recognition](https://mlsanity.com/paper/2407.02422/) (2024)
- [The Unreasonable Effectiveness of Pre-Trained Features for Camera Pose Refinement](https://mlsanity.com/paper/2404.10438/) (2024)
- [From Sparse to Dense: Camera Relocalization with Scene-Specific Detector from Feature Gaussian Splatting](https://mlsanity.com/paper/2503.19358/) (2025)
- [LoD-Loc: Aerial Visual Localization using LoD 3D Map with Neural Wireframe Alignment](https://mlsanity.com/paper/2410.12269/) (2024)
- [Learning to Produce Semi-dense Correspondences for Visual Localization](https://mlsanity.com/paper/2402.08359/) (2024)
- [A Guide to Structureless Visual Localization](https://mlsanity.com/paper/2504.17636/) (2025)
- [MeshVPR: Citywide Visual Place Recognition Using 3D Meshes](https://mlsanity.com/paper/2406.02776/) (2024)
- [LoD-Loc v2: Aerial Visual Localization over Low Level-of-Detail City Models using Explicit Silhouette Alignment](https://mlsanity.com/paper/2507.00659/) (2025)
- [Combining Absolute and Semi-Generalized Relative Poses for Visual Localization](https://mlsanity.com/paper/2409.14269/) (2024)
- [To Glue or Not to Glue? Classical vs Learned Image Matching for Mobile Mapping Cameras to Textured Semantic 3D Building Models](https://mlsanity.com/paper/2505.17973/) (2025)
- [Global-Aware Edge Prioritization for Pose Graph Initialization](https://mlsanity.com/paper/2602.21963/) (2026)
- [Scene Grounding In the Wild](https://mlsanity.com/paper/2603.26584/) (2026)
- [egenioussBench: A New Dataset for Geospatial Visual Localisation](https://mlsanity.com/paper/2605.05351/) (2026)
- [Map-Free Visual Relocalization Enhanced by Instance Knowledge and Depth Knowledge](https://mlsanity.com/paper/2408.13085/) (2024)
- [LoD-Loc v3: Generalized Aerial Localization in Dense Cities using Instance Silhouette Alignment](https://mlsanity.com/paper/2603.19609/) (2026)
- [PlanaReLoc: Camera Relocalization in 3D Planar Primitives via Region-Based Structure Matching](https://mlsanity.com/paper/2603.20818/) (2026)
- [Privacy-Preserving Structureless Visual Localization via Image Obfuscation](https://mlsanity.com/paper/2604.12068/) (2026)
- [SemCityLoc: Aerial 6DoF Localization Using Semantic 3D City Models](https://mlsanity.com/paper/2606.27444/) (2026)
- [Accuracy potential of visual localization exploiting high-end street-level imagery](https://mlsanity.com/paper/2607.24409/) (2026)

## Abstract (arXiv)

Visual localization is a core component in many applications, including augmented reality (AR). Localization algorithms compute the camera pose of a query image w.r.t. a scene representation, which is typically built from images. This often requires capturing and storing large amounts of data, followed by running Structure-from-Motion (SfM) algorithms. An interesting, and underexplored, source of data for building scene representations are 3D models that are readily available on the Internet, e.g., hand-drawn CAD models, 3D models generated from building footprints, or from aerial images. These models allow to perform visual localization right away without the time-consuming scene capturing and model building steps. Yet, it also comes with challenges as the available 3D models are often imperfect reflections of reality. E.g., the models might only have generic or no textures at all, might only provide a simple approximation of the scene geometry, or might be stretched. This paper studies how the imperfections of these models affect localization accuracy. We create a new benchmark for this task and provide a detailed experimental evaluation based on multiple 3D models per scene. We show that 3D models from the Internet show promise as an easy-to-obtain scene representation. At the same time, there is significant room for improvement for visual localization pipelines. To foster research on this interesting and challenging task, we release our benchmark at v-pnk.github.io/cadloc.