# Beyond Cross-view Image Retrieval: Highly Accurate Vehicle Localization Using Satellite Image

arXiv: https://arxiv.org/abs/2204.04752 · HTML: https://mlsanity.com/paper/2204.04752/

## Key facts

- Cited by 88 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 69 references that are in the map.
- First posted on arXiv 10 Apr 2022.

## Builds on (most cited first, 30 of 33)

- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [NetVLAD: CNN architecture for weakly supervised place recognition](https://mlsanity.com/paper/1511.07247/) (2015)
- [BARF: Bundle-Adjusting Neural Radiance Fields](https://mlsanity.com/paper/2104.06405/) (2021)
- [INeRF: Inverting Neural Radiance Fields for Pose Estimation](https://mlsanity.com/paper/2012.05877/) (2020)
- [DeepIM: Deep Iterative Matching for 6D Pose Estimation](https://mlsanity.com/paper/1804.00175/) (2018)
- [DeepView: View Synthesis with Learned Gradient Descent](https://mlsanity.com/paper/1906.07316/) (2019)
- [Large-Scale Image Retrieval with Attentive Deep Local Features](https://mlsanity.com/paper/1612.06321/) (2016)
- [Back to the Feature: Learning Robust Camera Localization from Pixels to Pose](https://mlsanity.com/paper/2103.09213/) (2021)
- [VIGOR: Cross-View Image Geo-localization beyond One-to-one Retrieval](https://mlsanity.com/paper/2011.12172/) (2020)
- [Wide-Area Image Geolocalization with Aerial Reference Imagery](https://mlsanity.com/paper/1510.03743/) (2015)
- [Lending Orientation to Neural Networks for Cross-view Geo-localization](https://mlsanity.com/paper/1903.12351/) (2019)
- [Where am I looking at? Joint Location and Orientation Estimation by Cross-View Matching](https://mlsanity.com/paper/2005.03860/) (2020)
- [Coming Down to Earth: Satellite-to-Street View Synthesis for Geo-Localization](https://mlsanity.com/paper/2103.06818/) (2021)
- [Self-supervising Fine-grained Region Similarities for Large-scale Image Localization](https://mlsanity.com/paper/2006.03926/) (2020)
- [Deep Learning Features at Scale for Visual Place Recognition](https://mlsanity.com/paper/1701.05105/) (2017)
- [Bridging the Domain Gap for Ground-to-Aerial Image Matching](https://mlsanity.com/paper/1904.11045/) (2019)
- [Localizing and Orienting Street Views Using Overhead Imagery](https://mlsanity.com/paper/1608.00161/) (2016)
- [Optimal Feature Transport for Cross-View Image Geo-Localization](https://mlsanity.com/paper/1907.05021/) (2019)
- [Predicting Ground-Level Scene Layout from Aerial Imagery](https://mlsanity.com/paper/1612.02709/) (2016)
- [Cross-View Image Matching for Geo-localization in Urban Environments](https://mlsanity.com/paper/1703.07815/) (2017)
- [Large-scale, real-time visual-inertial localization revisited](https://mlsanity.com/paper/1907.00338/) (2019)
- [Accurate 3-DoF Camera Geo-Localization via Ground-to-Satellite Image Matching](https://mlsanity.com/paper/2203.14148/) (2022)
- [Geometry-Guided Street-View Panorama Synthesis from Satellite Imagery](https://mlsanity.com/paper/2103.01623/) (2021)
- [Revisiting Street-to-Aerial View Image Geo-localization and Orientation Estimation](https://mlsanity.com/paper/2005.11592/) (2020)
- [Visual-Inertial Monocular SLAM with Map Reuse](https://mlsanity.com/paper/1610.05949/) (2016)
- [Semantic Cross-View Matching](https://mlsanity.com/paper/1511.00098/) (2015)
- [LM-Reloc: Levenberg-Marquardt Based Direct Visual Relocalization](https://mlsanity.com/paper/2010.06323/) (2020)
- [Image-Based Geo-Localization Using Satellite Imagery](https://mlsanity.com/paper/1903.00159/) (2019)
- [maplab: An Open Framework for Research in Visual-inertial Mapping and Localization](https://mlsanity.com/paper/1711.10250/) (2017)

## Built on by (most cited first, 30 of 88)

- [OrienterNet: Visual Localization in 2D Public Maps with Neural Matching](https://mlsanity.com/paper/2304.02009/) (2023)
- [Fine-Grained Cross-View Geo-Localization Using a Correlation-Aware Homography Estimator](https://mlsanity.com/paper/2308.16906/) (2023)
- [Uncertainty-aware Vision-based Metric Cross-view Geolocalization](https://mlsanity.com/paper/2211.12145/) (2022)
- [Visual Cross-View Metric Localization with Dense Uncertainty Estimates](https://mlsanity.com/paper/2208.08519/) (2022)
- [SliceMatch: Geometry-guided Aggregation for Cross-View Pose Estimation](https://mlsanity.com/paper/2211.14651/) (2022)
- [Boosting 3-DoF Ground-to-Satellite Camera Localization Accuracy via Geometry-Guided Cross-View Transformer](https://mlsanity.com/paper/2307.08015/) (2023)
- [Towards Natural Language-Guided Drones: GeoText-1652 Benchmark with Spatial Relation Matching](https://mlsanity.com/paper/2311.12751/) (2023)
- [AerialMegaDepth: Learning Aerial-Ground Reconstruction and View Synthesis](https://mlsanity.com/paper/2504.13157/) (2025)
- [SNAP: Self-Supervised Neural Maps for Visual Positioning and Semantic Understanding](https://mlsanity.com/paper/2306.05407/) (2023)
- [Convolutional Cross-View Pose Estimation](https://mlsanity.com/paper/2303.05915/) (2023)
- [Unleashing Unlabeled Data: A Paradigm for Cross-View Geo-Localization](https://mlsanity.com/paper/2403.14198/) (2024)
- [ObjectRelator: Enabling Cross-View Object Relation Understanding Across Ego-Centric and Exo-Centric Perspectives](https://mlsanity.com/paper/2411.19083/) (2024)
- [UrBench: A Comprehensive Benchmark for Evaluating Large Multimodal Models in Multi-View Urban Scenarios](https://mlsanity.com/paper/2408.17267/) (2024)
- [FG2: Fine-Grained Cross-View Localization by Fine-Grained Feature Matching](https://mlsanity.com/paper/2503.18725/) (2025)
- [CVLNet: Cross-View Semantic Correspondence Learning for Video-based Camera Localization](https://mlsanity.com/paper/2208.03660/) (2022)
- [Learning Dense Flow Field for Highly-accurate Cross-view Camera Localization](https://mlsanity.com/paper/2309.15556/) (2023)
- [Satellite Image Based Cross-view Localization for Autonomous Vehicle](https://mlsanity.com/paper/2207.13506/) (2022)
- [SG-BEV: Satellite-Guided BEV Fusion for Cross-View Semantic Segmentation](https://mlsanity.com/paper/2404.02638/) (2024)
- [View Consistent Purification for Accurate Cross-View Localization](https://mlsanity.com/paper/2308.08110/) (2023)
- [Cross-view geo-localization: a survey](https://mlsanity.com/paper/2406.09722/) (2024)
- [Weakly-supervised Camera Localization by Ground-to-satellite Image Registration](https://mlsanity.com/paper/2409.06471/) (2024)
- [Beyond Geo-localization: Fine-grained Orientation of Street-view Images by Cross-view Matching with Satellite Imagery with Supplementary Materials](https://mlsanity.com/paper/2307.03398/) (2023)
- [Composed Image Retrieval for Remote Sensing](https://mlsanity.com/paper/2405.15587/) (2024)
- [Adapting Fine-Grained Cross-View Localization to Areas without Fine Ground Truth](https://mlsanity.com/paper/2406.00474/) (2024)
- [LoD-Loc: Aerial Visual Localization using LoD 3D Map with Neural Wireframe Alignment](https://mlsanity.com/paper/2410.12269/) (2024)
- [Statewide Visual Geolocalization in the Wild](https://mlsanity.com/paper/2409.16763/) (2024)
- [GOMAA-Geo: GOal Modality Agnostic Active Geo-localization](https://mlsanity.com/paper/2406.01917/) (2024)
- [Multi-weather Cross-view Geo-localization Using Denoising Diffusion Models](https://mlsanity.com/paper/2408.02408/) (2024)
- [Cross-View Visual Geo-Localization for Outdoor Augmented Reality](https://mlsanity.com/paper/2303.15676/) (2023)
- [C-BEV: Contrastive Bird's Eye View Training for Cross-View Image Retrieval and 3-DoF Pose Estimation](https://mlsanity.com/paper/2312.08060/) (2023)

## Abstract (arXiv)

This paper addresses the problem of vehicle-mounted camera localization by matching a ground-level image with an overhead-view satellite map. Existing methods often treat this problem as cross-view image retrieval, and use learned deep features to match the ground-level query image to a partition (eg, a small patch) of the satellite map. By these methods, the localization accuracy is limited by the partitioning density of the satellite map (often in the order of tens meters). Departing from the conventional wisdom of image retrieval, this paper presents a novel solution that can achieve highly-accurate localization. The key idea is to formulate the task as pose estimation and solve it by neural-net based optimization. Specifically, we design a two-branch {CNN} to extract robust features from the ground and satellite images, respectively. To bridge the vast cross-view domain gap, we resort to a Geometry Projection module that projects features from the satellite map to the ground-view, based on a relative camera pose. Aiming to minimize the differences between the projected features and the observed features, we employ a differentiable Levenberg-Marquardt ({LM}) module to search for the optimal camera pose iteratively. The entire pipeline is differentiable and runs end-to-end. Extensive experiments on standard autonomous vehicle localization datasets have confirmed the superiority of the proposed method. Notably, e.g., starting from a coarse estimate of camera location within a wide region of 40m x 40m, with an 80% likelihood our method quickly reduces the lateral location error to be within 5m on a new KITTI cross-view dataset.