# Graph Attention Network for Camera Relocalization on Dynamic Scenes

arXiv: https://arxiv.org/abs/2209.15056 · HTML: https://mlsanity.com/paper/2209.15056/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 32 references that are in the map.
- First posted on arXiv 29 Sep 2022.

## Builds on (most cited first, 17 of 17)

- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [NetVLAD: CNN architecture for weakly supervised place recognition](https://mlsanity.com/paper/1511.07247/) (2015)
- [From Coarse to Fine: Robust Hierarchical Localization at Large Scale](https://mlsanity.com/paper/1812.03506/) (2018)
- [DSAC - Differentiable RANSAC for Camera Localization](https://mlsanity.com/paper/1611.05705/) (2016)
- [Geometric Loss Functions for Camera Pose Regression with Deep Learning](https://mlsanity.com/paper/1704.00390/) (2017)
- [Learning Less is More - 6D Camera Localization via 3D Surface Regression](https://mlsanity.com/paper/1711.10228/) (2017)
- [Robust Image Retrieval-based Visual Localization using Kapture](https://mlsanity.com/paper/2007.13867/) (2020)
- [Image-based Localization using Hourglass Networks](https://mlsanity.com/paper/1703.07971/) (2017)
- [VidLoc: A Deep Spatio-Temporal Model for 6-DoF Video-Clip Relocalization](https://mlsanity.com/paper/1702.06521/) (2017)
- [On-the-Fly Adaptation of Regression Forests for Online Camera Relocalisation](https://mlsanity.com/paper/1702.02779/) (2017)
- [Real-Time RGB-D Camera Pose Estimation in Novel Scenes using a Relocalisation Cascade](https://mlsanity.com/paper/1810.12163/) (2018)
- [Beyond Controlled Environments: 3D Camera Re-Localization in Changing Indoor Scenes](https://mlsanity.com/paper/2008.02004/) (2020)
- [Let's Take This Online: Adapting Scene Coordinate Regression Network Predictions for Online RGB-D Camera Relocalisation](https://mlsanity.com/paper/1906.08744/) (2019)
- [Learning Camera Localization via Dense Scene Matching](https://mlsanity.com/paper/2103.16792/) (2021)
- [Robust Neural Routing Through Space Partitions for Camera Relocalization in Dynamic Indoor Environments](https://mlsanity.com/paper/2012.04746/) (2020)
- [Collaborative Large-Scale Dense 3D Reconstruction with Online Inter-Agent Pose Optimisation](https://mlsanity.com/paper/1801.08361/) (2018)

## Abstract (arXiv)

We devise a graph attention network-based approach for learning a scene triangle mesh representation in order to estimate an image camera position in a dynamic environment. Previous approaches built a scene-dependent model that explicitly or implicitly embeds the structure of the scene. They use convolution neural networks or decision trees to establish 2D/3D-3D correspondences. Such a mapping overfits the target scene and does not generalize well to dynamic changes in the environment. Our work introduces a novel approach to solve the camera relocalization problem by using the available triangle mesh. Our 3D-3D matching framework consists of three blocks: (1) a graph neural network to compute the embedding of mesh vertices, (2) a convolution neural network to compute the embedding of grid cells defined on the RGB-D image, and (3) a neural network model to establish the correspondence between the two embeddings. These three components are trained end-to-end. To predict the final pose, we run the RANSAC algorithm to generate camera pose hypotheses, and we refine the prediction using the point-cloud representation. Our approach significantly improves the camera pose accuracy of the state-of-the-art method from 0.358 to 0.506 on the RIO10 benchmark for dynamic indoor camera relocalization.