# Continuous Self-Localization on Aerial Images Using Visual and Lidar Sensors

arXiv: https://arxiv.org/abs/2203.03334 · HTML: https://mlsanity.com/paper/2203.03334/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 56 references that are in the map.
- First posted on arXiv 7 Mar 2022.

## Builds on (most cited first, 26 of 26)

- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [Unified Perceptual Parsing for Scene Understanding](https://mlsanity.com/paper/1807.10221/) (2018)
- [Semantic Understanding of Scenes through the ADE20K Dataset](https://mlsanity.com/paper/1608.05442/) (2016)
- [On the Variance of the Adaptive Learning Rate and Beyond](https://mlsanity.com/paper/1908.03265/) (2019)
- [Argoverse: 3D Tracking and Forecasting with Rich Maps](https://mlsanity.com/paper/1911.02620/) (2019)
- [Training Region-based Object Detectors with Online Hard Example Mining](https://mlsanity.com/paper/1604.03540/) (2016)
- [KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D](https://mlsanity.com/paper/2109.13410/) (2021)
- [One Thousand and One Hours: Self-driving Motion Prediction Dataset](https://mlsanity.com/paper/2006.14480/) (2020)
- [PandaSet: Advanced Sensor Suite Dataset for Autonomous Driving](https://mlsanity.com/paper/2112.12610/) (2021)
- [Wide-Area Image Geolocalization with Aerial Reference Imagery](https://mlsanity.com/paper/1510.03743/) (2015)
- [Lending Orientation to Neural Networks for Cross-view Geo-localization](https://mlsanity.com/paper/1903.12351/) (2019)
- [The Oxford Radar RobotCar Dataset: A Radar Extension to the Oxford RobotCar Dataset](https://mlsanity.com/paper/1909.01300/) (2019)
- [Each Part Matters: Local Patterns Facilitate Cross-view Geo-localization](https://mlsanity.com/paper/2008.11646/) (2020)
- [Ford Multi-AV Seasonal Dataset](https://mlsanity.com/paper/2003.07969/) (2020)
- [Project AutoVision: Localization and 3D Scene Perception for an Autonomous Vehicle with a Multi-Camera System](https://mlsanity.com/paper/1809.05477/) (2018)
- [RSL-Net: Localising in Satellite Images From a Radar on the Ground](https://mlsanity.com/paper/2001.03233/) (2020)
- [Masking by Moving: Learning Distraction-Free Radar Odometry from Pose Information](https://mlsanity.com/paper/1909.03752/) (2019)
- [Aerial Imagery based LIDAR Localization for Autonomous Vehicles](https://mlsanity.com/paper/2003.11192/) (2020)
- [Accurate Vision-based Vehicle Localization using Satellite Imagery](https://mlsanity.com/paper/1510.09171/) (2015)
- [Complex Urban LiDAR Data Set](https://mlsanity.com/paper/1803.06121/) (2018)
- [Map-Based Temporally Consistent Geolocalization through Learning Motion Trajectories](https://mlsanity.com/paper/2010.06117/) (2020)

## Built on by (most cited first, 10 of 10)

- [Uncertainty-aware Vision-based Metric Cross-view Geolocalization](https://mlsanity.com/paper/2211.12145/) (2022)
- [Satellite Image Based Cross-view Localization for Autonomous Vehicle](https://mlsanity.com/paper/2207.13506/) (2022)
- [View Consistent Purification for Accurate Cross-View Localization](https://mlsanity.com/paper/2308.08110/) (2023)
- [C-BEV: Contrastive Bird's Eye View Training for Cross-View Image Retrieval and 3-DoF Pose Estimation](https://mlsanity.com/paper/2312.08060/) (2023)
- [PIDLoc: Cross-View Pose Optimization Network Inspired by PID Controllers](https://mlsanity.com/paper/2503.02388/) (2025)
- [View-Invariant Localization using Semantic Objects in Changing Environments](https://mlsanity.com/paper/2209.14426/) (2022)
- [AGL-NET: Aerial-Ground Cross-Modal Global Localization with Varying Scales](https://mlsanity.com/paper/2404.03187/) (2024)
- [RaSCL: Radar to Satellite Crossview Localization](https://mlsanity.com/paper/2504.15899/) (2025)
- [Geo-Localization Based on Dynamically Weighted Factor-Graph](https://mlsanity.com/paper/2311.07301/) (2023)
- [TACO: Trajectory Aligning Cross-view Optimisation](https://mlsanity.com/paper/2605.03315/) (2026)

## Abstract (arXiv)

This paper proposes a novel method for geo-tracking, i.e. continuous metric self-localization in outdoor environments by registering a vehicle's sensor information with aerial imagery of an unseen target region. Geo-tracking methods offer the potential to supplant noisy signals from global navigation satellite systems (GNSS) and expensive and hard to maintain prior maps that are typically used for this purpose. The proposed geo-tracking method aligns data from on-board cameras and lidar sensors with geo-registered orthophotos to continuously localize a vehicle. We train a model in a metric learning setting to extract visual features from ground and aerial images. The ground features are projected into a top-down perspective via the lidar points and are matched with the aerial features to determine the relative pose between vehicle and orthophoto. Our method is the first to utilize on-board cameras in an end-to-end differentiable model for metric self-localization on unseen orthophotos. It exhibits strong generalization, is robust to changes in the environment and requires only geo-poses as ground truth. We evaluate our approach on the KITTI-360 dataset and achieve a mean absolute position error (APE) of 0.94m. We further compare with previous approaches on the KITTI odometry dataset and achieve state-of-the-art results on the geo-tracking task.