# OrienterNet: Visual Localization in 2D Public Maps with Neural Matching

arXiv: https://arxiv.org/abs/2304.02009 · HTML: https://mlsanity.com/paper/2304.02009/

## Key facts

- Cited by 84 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 85 references that are in the map.
- First posted on arXiv 4 Apr 2023.

## Builds on (most cited first, 30 of 34)

- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3D](https://mlsanity.com/paper/2008.05711/) (2020)
- [NetVLAD: CNN architecture for weakly supervised place recognition](https://mlsanity.com/paper/1511.07247/) (2015)
- [From Coarse to Fine: Robust Hierarchical Localization at Large Scale](https://mlsanity.com/paper/1812.03506/) (2018)
- [DISK: Learning local features with policy gradient](https://mlsanity.com/paper/2006.13566/) (2020)
- [R2D2: Repeatable and Reliable Detector and Descriptor](https://mlsanity.com/paper/1906.06195/) (2019)
- [Pixel-Perfect Structure-from-Motion with Featuremetric Refinement](https://mlsanity.com/paper/2108.08291/) (2021)
- [Back to the Feature: Learning Robust Camera Localization from Pixels to Pose](https://mlsanity.com/paper/2103.09213/) (2021)
- [VIGOR: Cross-View Image Geo-localization beyond One-to-one Retrieval](https://mlsanity.com/paper/2011.12172/) (2020)
- [Predicting Semantic Map Representations from Images using Pyramid Occupancy Networks](https://mlsanity.com/paper/2003.13402/) (2020)
- [Where am I looking at? Joint Location and Orientation Estimation by Cross-View Matching](https://mlsanity.com/paper/2005.03860/) (2020)
- [Translating Images into Maps](https://mlsanity.com/paper/2110.00966/) (2021)
- [Optimal Feature Transport for Cross-View Image Geo-Localization](https://mlsanity.com/paper/1907.05021/) (2019)
- [Beyond Cross-view Image Retrieval: Highly Accurate Vehicle Localization Using Satellite Image](https://mlsanity.com/paper/2204.04752/) (2022)
- [Revealing Scenes by Inverting Structure from Motion Reconstructions](https://mlsanity.com/paper/1904.03303/) (2019)
- [Three Ways to Improve Semantic Segmentation with Self-Supervised Depth Estimation](https://mlsanity.com/paper/2012.10782/) (2020)
- [Is Geometry Enough for Matching in Visual Localization?](https://mlsanity.com/paper/2203.12979/) (2022)
- [Visual Cross-View Metric Localization with Dense Uncertainty Estimates](https://mlsanity.com/paper/2208.08519/) (2022)
- [Privacy Preserving Image-Based Localization](https://mlsanity.com/paper/1903.05572/) (2019)
- [Large-scale, real-time visual-inertial localization revisited](https://mlsanity.com/paper/1907.00338/) (2019)
- [Hybrid Scene Compression for Visual Localization](https://mlsanity.com/paper/1807.07512/) (2018)
- [Semantic Cross-View Matching](https://mlsanity.com/paper/1511.00098/) (2015)
- [LaLaLoc: Latent Layout Localisation in Dynamic, Unvisited Environments](https://mlsanity.com/paper/2104.09169/) (2021)
- [Learning to Localize Using a LiDAR Intensity Map](https://mlsanity.com/paper/2012.10902/) (2020)
- [Exploiting Sparse Semantic HD Maps for Self-Driving Vehicle Localization](https://mlsanity.com/paper/1908.03274/) (2019)
- [LASER: LAtent SpacE Rendering for 2D Visual Localization](https://mlsanity.com/paper/2204.00157/) (2022)
- [NinjaDesc: Content-Concealing Visual Descriptors via Adversarial Learning](https://mlsanity.com/paper/2112.12785/) (2021)
- [You Are Here: Geolocation by Embedding Maps and Images](https://mlsanity.com/paper/1911.08797/) (2019)
- [Masking by Moving: Learning Distraction-Free Radar Odometry from Pose Information](https://mlsanity.com/paper/1909.03752/) (2019)

## Built on by (most cited first, 30 of 84)

- [An Outlook into the Future of Egocentric Vision](https://mlsanity.com/paper/2308.07123/) (2023)
- [Fine-Grained Cross-View Geo-Localization Using a Correlation-Aware Homography Estimator](https://mlsanity.com/paper/2308.16906/) (2023)
- [P-MapNet: Far-seeing Map Generator Enhanced by both SDMap and HDMap Priors](https://mlsanity.com/paper/2403.10521/) (2024)
- [Cross-view image geo-localization with Panorama-BEV Co-Retrieval Network](https://mlsanity.com/paper/2408.05475/) (2024)
- [SNAP: Self-Supervised Neural Maps for Visual Positioning and Semantic Understanding](https://mlsanity.com/paper/2306.05407/) (2023)
- [Text2Loc: 3D Point Cloud Localization from Natural Language](https://mlsanity.com/paper/2311.15977/) (2023)
- [GraphAlign: Enhancing Accurate Feature Alignment by Graph matching for Multi-Modal 3D Object Detection](https://mlsanity.com/paper/2310.08261/) (2023)
- [FreeReg: Image-to-Point Cloud Registration Leveraging Pretrained Diffusion Models and Monocular Depth Estimators](https://mlsanity.com/paper/2310.03420/) (2023)
- [SceneGraphLoc: Cross-Modal Coarse Visual Localization on 3D Scene Graphs](https://mlsanity.com/paper/2404.00469/) (2024)
- [F3Loc: Fusion and Filtering for Floorplan Localization](https://mlsanity.com/paper/2403.03370/) (2024)
- [MapLocNet: Coarse-to-Fine Feature Registration for Visual Re-Localization in Navigation Maps](https://mlsanity.com/paper/2407.08561/) (2024)
- [Weakly-supervised Camera Localization by Ground-to-satellite Image Registration](https://mlsanity.com/paper/2409.06471/) (2024)
- [Adapting Fine-Grained Cross-View Localization to Areas without Fine Ground Truth](https://mlsanity.com/paper/2406.00474/) (2024)
- [U-BEV: Height-aware Bird's-Eye-View Segmentation and Neural Map-based Relocalization](https://mlsanity.com/paper/2310.13766/) (2023)
- [LoD-Loc: Aerial Visual Localization using LoD 3D Map with Neural Wireframe Alignment](https://mlsanity.com/paper/2410.12269/) (2024)
- [BLOS-BEV: Navigation Map Enhanced Lane Segmentation Network, Beyond Line of Sight](https://mlsanity.com/paper/2407.08526/) (2024)
- [Statewide Visual Geolocalization in the Wild](https://mlsanity.com/paper/2409.16763/) (2024)
- [Leveraging BEV Paradigm for Ground-to-Aerial Image Synthesis](https://mlsanity.com/paper/2408.01812/) (2024)
- [C-BEV: Contrastive Bird's Eye View Training for Cross-View Image Retrieval and 3-DoF Pose Estimation](https://mlsanity.com/paper/2312.08060/) (2023)
- [Scaling Image Geo-Localization to Continent Level](https://mlsanity.com/paper/2510.26795/) (2025)
- [Cross-View Meets Diffusion: Aerial Image Synthesis with Geometry and Text Guidance](https://mlsanity.com/paper/2408.04224/) (2024)
- [Seeing through Satellite Images at Street Views](https://mlsanity.com/paper/2505.17001/) (2025)
- [BevSplat: Resolving Height Ambiguity via Feature-Based Gaussian Primitives for Weakly-Supervised Cross-View Localization](https://mlsanity.com/paper/2502.09080/) (2025)
- [GeoDistill: Geometry-Guided Self-Distillation for Weakly Supervised Cross-View Localization](https://mlsanity.com/paper/2507.10935/) (2025)
- [UnLoc: Leveraging Depth Uncertainties for Floorplan Localization](https://mlsanity.com/paper/2509.11301/) (2025)
- [SegLocNet: Multimodal Localization Network for Autonomous Driving via Bird's-Eye-View Segmentation](https://mlsanity.com/paper/2502.20077/) (2025)
- [Map It Anywhere (MIA): Empowering Bird's Eye View Mapping using Large-scale Public Data](https://mlsanity.com/paper/2407.08726/) (2024)
- [Geometry-guided Cross-view Diffusion for One-to-many Cross-view Image Synthesis](https://mlsanity.com/paper/2412.03315/) (2024)
- [LoD-Loc v2: Aerial Visual Localization over Low Level-of-Detail City Models using Explicit Silhouette Alignment](https://mlsanity.com/paper/2507.00659/) (2025)
- [U-ViLAR: Uncertainty-Aware Visual Localization for Autonomous Driving via Differentiable Association and Registration](https://mlsanity.com/paper/2507.04503/) (2025)

## Abstract (arXiv)

Humans can orient themselves in their 3D environments using simple 2D maps. Differently, algorithms for visual localization mostly rely on complex 3D point clouds that are expensive to build, store, and maintain over time. We bridge this gap by introducing OrienterNet, the first deep neural network that can localize an image with sub-meter accuracy using the same 2D semantic maps that humans use. OrienterNet estimates the location and orientation of a query image by matching a neural Bird's-Eye View with open and globally available maps from OpenStreetMap, enabling anyone to localize anywhere such maps are available. OrienterNet is supervised only by camera poses but learns to perform semantic matching with a wide range of map elements in an end-to-end manner. To enable this, we introduce a large crowd-sourced dataset of images captured across 12 cities from the diverse viewpoints of cars, bikes, and pedestrians. OrienterNet generalizes to new datasets and pushes the state of the art in both robotics and AR scenarios. The code and trained model will be released publicly.