# SPVLoc: Semantic Panoramic Viewport Matching for 6D Camera Localization in Unseen Environments

arXiv: https://arxiv.org/abs/2404.10527 · HTML: https://mlsanity.com/paper/2404.10527/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 50 references that are in the map.
- First posted on arXiv 16 Apr 2024.

## Builds on (most cited first, 18 of 18)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [NetVLAD: CNN architecture for weakly supervised place recognition](https://mlsanity.com/paper/1511.07247/) (2015)
- [Structured3D: A Large Photo-realistic Dataset for Structured 3D Modeling](https://mlsanity.com/paper/1908.00222/) (2019)
- [Geometric Loss Functions for Camera Pose Regression with Deep Learning](https://mlsanity.com/paper/1704.00390/) (2017)
- [Back to the Feature: Learning Robust Camera Localization from Pixels to Pose](https://mlsanity.com/paper/2103.09213/) (2021)
- [Particle Filter Networks with Application to Visual Localization](https://mlsanity.com/paper/1805.08975/) (2018)
- [Corners for Layout: End-to-End Layout Recovery from 360 Images](https://mlsanity.com/paper/1903.08094/) (2019)
- [Visual Camera Re-Localization Using Graph Neural Networks and Relative Pose Supervision](https://mlsanity.com/paper/2104.02538/) (2021)
- [LaLaLoc: Latent Layout Localisation in Dynamic, Unvisited Environments](https://mlsanity.com/paper/2104.09169/) (2021)
- [LASER: LAtent SpacE Rendering for 2D Visual Localization](https://mlsanity.com/paper/2204.00157/) (2022)
- [F3Loc: Fusion and Filtering for Floorplan Localization](https://mlsanity.com/paper/2403.03370/) (2024)
- [Target Driven Instance Detection](https://mlsanity.com/paper/1803.04610/) (2018)
- [Fully Geometric Panoramic Localization](https://mlsanity.com/paper/2403.19904/) (2024)
- [Semantic Pose Verification for Outdoor Visual Localization with Self-supervised Contrastive Learning](https://mlsanity.com/paper/2203.16945/) (2022)
- [PanoVPR: Towards Unified Perspective-to-Equirectangular Visual Place Recognition via Sliding Windows across the Panoramic View](https://mlsanity.com/paper/2303.14095/) (2023)
- [Automatic Reconstruction of Semantic 3D Models from 2D Floor Plans](https://mlsanity.com/paper/2306.01642/) (2023)
- [Zero in on Shape: A Generic 2D-3D Instance Similarity Metric learned from Synthetic Data](https://mlsanity.com/paper/2108.04091/) (2021)

## Built on by (most cited first, 5 of 5)

- [Supercharging Floorplan Localization with Semantic Rays](https://mlsanity.com/paper/2507.09291/) (2025)
- [Z-FLoc: Zero-Shot Floorplan Localization via Geometric Primitives](https://mlsanity.com/paper/2606.04788/) (2026)
- [Towards Global Localization using Multi-Modal Object-Instance Re-Identification](https://mlsanity.com/paper/2409.12002/) (2024)
- [Automatic Drywall Analysis for Progress Tracking and Quality Control in Construction](https://mlsanity.com/paper/2503.03422/) (2025)
- [PlanaReLoc: Camera Relocalization in 3D Planar Primitives via Region-Based Structure Matching](https://mlsanity.com/paper/2603.20818/) (2026)

## Abstract (arXiv)

In this paper, we present SPVLoc, a global indoor localization method that accurately determines the six-dimensional (6D) camera pose of a query image and requires minimal scene-specific prior knowledge and no scene-specific training. Our approach employs a novel matching procedure to localize the perspective camera's viewport, given as an RGB image, within a set of panoramic semantic layout representations of the indoor environment. The panoramas are rendered from an untextured 3D reference model, which only comprises approximate structural information about room shapes, along with door and window annotations. We demonstrate that a straightforward convolutional network structure can successfully achieve image-to-panorama and ultimately image-to-model matching. Through a viewport classification score, we rank reference panoramas and select the best match for the query image. Then, a 6D relative pose is estimated between the chosen panorama and query image. Our experiments demonstrate that this approach not only efficiently bridges the domain gap but also generalizes well to previously unseen scenes that are not part of the training data. Moreover, it achieves superior localization accuracy compared to the state of the art methods and also estimates more degrees of freedom of the camera pose. Our source code is publicly available at https://fraunhoferhhi.github.io/spvloc .