# FusionLoc: Camera-2D LiDAR Fusion Using Multi-Head Self-Attention for End-to-End Serving Robot Relocalization

arXiv: https://arxiv.org/abs/2303.06872 · HTML: https://mlsanity.com/paper/2303.06872/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 47 references that are in the map.
- First posted on arXiv 13 Mar 2023.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [Dual Attention Network for Scene Segmentation](https://mlsanity.com/paper/1809.02983/) (2018)
- [NetVLAD: CNN architecture for weakly supervised place recognition](https://mlsanity.com/paper/1511.07247/) (2015)
- [Geometric Loss Functions for Camera Pose Regression with Deep Learning](https://mlsanity.com/paper/1704.00390/) (2017)
- [Geometry-Aware Learning of Maps for Camera Localization](https://mlsanity.com/paper/1712.03342/) (2017)
- [Image-based localization using LSTMs for structured feature correlation](https://mlsanity.com/paper/1611.07890/) (2016)
- [Modelling Uncertainty in Deep Learning for Camera Relocalization](https://mlsanity.com/paper/1509.05909/) (2015)
- [Deformable Siamese Attention Networks for Visual Object Tracking](https://mlsanity.com/paper/2004.06711/) (2020)
- [AtLoc: Attention Guided Camera Localization](https://mlsanity.com/paper/1909.03557/) (2019)
- [Camera Relocalization by Computing Pairwise Relative Poses Using Convolutional Neural Network](https://mlsanity.com/paper/1707.09733/) (2017)
- [RoNIN: Robust Neural Inertial Navigation in the Wild: Benchmark, Evaluations, and New Methods](https://mlsanity.com/paper/1905.12853/) (2019)
- [Image-based Localization using Hourglass Networks](https://mlsanity.com/paper/1703.07971/) (2017)
- [VidLoc: A Deep Spatio-Temporal Model for 6-DoF Video-Clip Relocalization](https://mlsanity.com/paper/1702.06521/) (2017)
- [MinkLoc++: Lidar and Monocular Image Fusion for Place Recognition](https://mlsanity.com/paper/2104.05327/) (2021)
- [IDOL: Inertial Deep Orientation-Estimation and Localization](https://mlsanity.com/paper/2102.04024/) (2021)
- [PointLoc: Deep Pose Regressor for LiDAR Point Cloud Localization](https://mlsanity.com/paper/2003.02392/) (2020)
- [Augmenting Visual Place Recognition with Structural Cues](https://mlsanity.com/paper/2003.00278/) (2020)
- [CORAL: Colored structural representation for bi-modal place recognition](https://mlsanity.com/paper/2011.10934/) (2020)
- [DeepPCO: End-to-End Point Cloud Odometry through Deep Parallel Neural Network](https://mlsanity.com/paper/1910.11088/) (2019)
- [VMLoc: Variational Fusion For Learning-Based Multimodal Camera Localization](https://mlsanity.com/paper/2003.07289/) (2020)
- [Neural Inertial Localization](https://mlsanity.com/paper/2203.15851/) (2022)

## Built on by (most cited first, 2 of 2)

- [Localization Meets Uncertainty: Uncertainty-Aware Multi-Modal Localization](https://mlsanity.com/paper/2504.07677/) (2025)
- [MVL-Loc: Leveraging Vision-Language Model for Generalizable Multi-Scene Camera Relocalization](https://mlsanity.com/paper/2507.04509/) (2025)

## Abstract (arXiv)

As technology advances in autonomous mobile robots, mobile service robots have been actively used more and more for various purposes. Especially, serving robots have been not surprising products anymore since the COVID-19 pandemic. One of the practical problems in operating a serving robot is that it often fails to estimate its pose on a map that it moves around. Whenever the failure happens, servers should bring the serving robot to its initial location and reboot it manually. In this paper, we focus on end-to-end relocalization of serving robots to address the problem. It is to predict robot pose directly from only the onboard sensor data using neural networks. In particular, we propose a deep neural network architecture for the relocalization based on camera-2D LiDAR sensor fusion. We call the proposed method FusionLoc. In the proposed method, the multi-head self-attention complements different types of information captured by the two sensors to regress the robot pose. Our experiments on a dataset collected by a commercial serving robot demonstrate that FusionLoc can provide better performances than previous end-to-end relocalization methods taking only a single image or a 2D LiDAR point cloud as well as a straightforward fusion method concatenating their features.