# CPO: Change Robust Panorama to Point Cloud Localization

arXiv: https://arxiv.org/abs/2207.05317 · HTML: https://mlsanity.com/paper/2207.05317/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 41 references that are in the map.
- First posted on arXiv 12 Jul 2022.

## Builds on (most cited first, 21 of 21)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [NetVLAD: CNN architecture for weakly supervised place recognition](https://mlsanity.com/paper/1511.07247/) (2015)
- [From Coarse to Fine: Robust Hierarchical Localization at Large Scale](https://mlsanity.com/paper/1812.03506/) (2018)
- [Joint 2D-3D-Semantic Data for Indoor Scene Understanding](https://mlsanity.com/paper/1702.01105/) (2017)
- [Structured3D: A Large Photo-realistic Dataset for Structured 3D Modeling](https://mlsanity.com/paper/1908.00222/) (2019)
- [Deep Photo Style Transfer](https://mlsanity.com/paper/1703.07511/) (2017)
- [Image-based localization using LSTMs for structured feature correlation](https://mlsanity.com/paper/1611.07890/) (2016)
- [Patch2Pix: Epipolar-Guided Pixel-Level Correspondences](https://mlsanity.com/paper/2012.01909/) (2020)
- [Self-supervising Fine-grained Region Similarities for Large-scale Image Localization](https://mlsanity.com/paper/2006.03926/) (2020)
- [Hierarchical Scene Coordinate Classification and Regression for Visual Localization](https://mlsanity.com/paper/1909.06216/) (2019)
- [Robust Image Retrieval-based Visual Localization using Kapture](https://mlsanity.com/paper/2007.13867/) (2020)
- [Pano3D: A Holistic Benchmark and a Solid Baseline for 360o Depth Estimation](https://mlsanity.com/paper/2109.02749/) (2021)
- [Beyond Controlled Environments: 3D Camera Re-Localization in Changing Indoor Scenes](https://mlsanity.com/paper/2008.02004/) (2020)
- [Cross-Camera Convolutional Color Constancy](https://mlsanity.com/paper/2011.11890/) (2020)
- [LaLaLoc: Latent Layout Localisation in Dynamic, Unvisited Environments](https://mlsanity.com/paper/2104.09169/) (2021)
- [Real-time Kinematic Ground Truth for the Oxford RobotCar Dataset](https://mlsanity.com/paper/2002.10152/) (2020)
- [The Alignment of the Spheres: Globally-Optimal Spherical Mixture Alignment for Camera Pose Estimation](https://mlsanity.com/paper/1812.01232/) (2018)
- [Robust Neural Routing Through Space Partitions for Camera Relocalization in Dynamic Indoor Environments](https://mlsanity.com/paper/2012.04746/) (2020)
- [PICCOLO: Point Cloud-Centric Omnidirectional Localization](https://mlsanity.com/paper/2108.06545/) (2021)

## Built on by (most cited first, 3 of 3)

- [360Loc: A Dataset and Benchmark for Omnidirectional Visual Localization with Cross-device Queries](https://mlsanity.com/paper/2311.17389/) (2023)
- [Fully Geometric Panoramic Localization](https://mlsanity.com/paper/2403.19904/) (2024)
- [LDL: Line Distance Functions for Panoramic Localization](https://mlsanity.com/paper/2308.13989/) (2023)

## Abstract (arXiv)

We present CPO, a fast and robust algorithm that localizes a 2D panorama with respect to a 3D point cloud of a scene possibly containing changes. To robustly handle scene changes, our approach deviates from conventional feature point matching, and focuses on the spatial context provided from panorama images. Specifically, we propose efficient color histogram generation and subsequent robust localization using score maps. By utilizing the unique equivariance of spherical projections, we propose very fast color histogram generation for a large number of camera poses without explicitly rendering images for all candidate poses. We accumulate the regional consistency of the panorama and point cloud as 2D/3D score maps, and use them to weigh the input color values to further increase robustness. The weighted color distribution quickly finds good initial poses and achieves stable convergence for gradient-based optimization. CPO is lightweight and achieves effective localization in all tested scenarios, showing stable performance despite scene changes, repetitive structures, or featureless regions, which are typical challenges for visual localization with perspective cameras. Code is available at urlhttps://github.com/82magnolia/panoramic-localization/.