# View-Invariant Localization using Semantic Objects in Changing Environments

arXiv: https://arxiv.org/abs/2209.14426 · HTML: https://mlsanity.com/paper/2209.14426/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 50 references that are in the map.
- First posted on arXiv 28 Sep 2022.

## Builds on (most cited first, 20 of 20)

- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection](https://mlsanity.com/paper/1711.06396/) (2017)
- [ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM](https://mlsanity.com/paper/2007.11898/) (2020)
- [Oriented RepPoints for Aerial Object Detection](https://mlsanity.com/paper/2105.11111/) (2021)
- [Each Part Matters: Local Patterns Facilitate Cross-view Geo-localization](https://mlsanity.com/paper/2008.11646/) (2020)
- [Cross-View Image Matching for Geo-localization in Urban Environments](https://mlsanity.com/paper/1703.07815/) (2017)
- [Learning RoI Transformer for Detecting Oriented Objects in Aerial Images](https://mlsanity.com/paper/1812.00155/) (2018)
- [CLIPPER: A Graph-Theoretic Framework for Robust Data Association](https://mlsanity.com/paper/2011.10202/) (2020)
- [Learning to Localize Using a LiDAR Intensity Map](https://mlsanity.com/paper/2012.10902/) (2020)
- [Satellite Image-based Localization via Learned Embeddings](https://mlsanity.com/paper/1704.01133/) (2017)
- [Parallel Maximum Clique Algorithms with Applications to Network Analysis and Storage](https://mlsanity.com/paper/1302.6256/) (2013)
- [Long-term Visual Localization using Semantically Segmented Images](https://mlsanity.com/paper/1801.05269/) (2018)
- [City-wide Street-to-Satellite Image Geolocalization of a Mobile Ground Agent](https://mlsanity.com/paper/2203.05612/) (2022)
- [Any Way You Look At It: Semantic Crossview Localization and Mapping with LiDAR](https://mlsanity.com/paper/2203.08925/) (2022)
- [Aerial Imagery based LIDAR Localization for Autonomous Vehicles](https://mlsanity.com/paper/2003.11192/) (2020)
- [Continuous Self-Localization on Aerial Images Using Visual and Lidar Sensors](https://mlsanity.com/paper/2203.03334/) (2022)
- [Global Data Association for SLAM with 3D Grassmannian Manifold Objects](https://mlsanity.com/paper/2205.08556/) (2022)
- [3D Registration of Aerial and Ground Robots for Disaster Response: An Evaluation of Features, Descriptors, and Transformation Estimation](https://mlsanity.com/paper/1709.00587/) (2017)

## Built on by (most cited first, 2 of 2)

- [CLIPPER+: A Fast Maximal Clique Algorithm for Robust Global Registration](https://mlsanity.com/paper/2402.15464/) (2024)
- [SWA-PF: Semantic-Weighted Adaptive Particle Filter for Memory-Efficient 4-DoF UAV Localization in GNSS-Denied Environments](https://mlsanity.com/paper/2509.13795/) (2025)

## Abstract (arXiv)

This paper proposes a novel framework for real-time localization and egomotion tracking of a vehicle in a reference map. The core idea is to map the semantic objects observed by the vehicle and register them to their corresponding objects in the reference map. While several recent works have leveraged semantic information for cross-view localization, the main contribution of this work is a view-invariant formulation that makes the approach directly applicable to any viewpoint configuration for which objects are detectable. Another distinctive feature is robustness to changes in the environment/objects due to a data association scheme suited for extreme outlier regimes (e.g., 90% association outliers). To demonstrate our framework, we consider an example of localizing a ground vehicle in a reference object map using only cars as objects. While only a stereo camera is used for the ground vehicle, we consider reference maps constructed a priori from ground viewpoints using stereo cameras and Lidar scans, and georeferenced aerial images captured at a different date to demonstrate the framework's robustness to different modalities, viewpoints, and environment changes. Evaluations on the KITTI dataset show that over a 3.7 km trajectory, localization occurs in 36 sec and is followed by real-time egomotion tracking with an average position error of 8.5 m in a Lidar reference map, and on an aerial object map where 77% of objects are outliers, localization is achieved in 71 sec with an average position error of 7.9 m.