# External Camera-based Mobile Robot Pose Estimation for Collaborative Perception with Smart Edge Sensors

arXiv: https://arxiv.org/abs/2303.03797 · HTML: https://mlsanity.com/paper/2303.03797/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 40 references that are in the map.
- First posted on arXiv 7 Mar 2023.

## Builds on (most cited first, 21 of 21)

- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes](https://mlsanity.com/paper/1711.00199/) (2017)
- [Simple Baselines for Human Pose Estimation and Tracking](https://mlsanity.com/paper/1804.06208/) (2018)
- [PVNet: Pixel-wise Voting Network for 6DoF Pose Estimation](https://mlsanity.com/paper/1812.11788/) (2018)
- [DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion](https://mlsanity.com/paper/1901.04780/) (2019)
- [BB8: A Scalable, Accurate, Robust to Partial Occlusion Method for Predicting the 3D Poses of Challenging Objects without Using Depth](https://mlsanity.com/paper/1703.10896/) (2017)
- [Markerless tracking of user-defined features with deep learning](https://mlsanity.com/paper/1804.03142/) (2018)
- [MobileDets: Searching for Object Detection Architectures for Mobile Accelerators](https://mlsanity.com/paper/2004.14525/) (2020)
- [Camera-to-Robot Pose Estimation from a Single Image](https://mlsanity.com/paper/1911.09231/) (2019)
- [X-View: Graph-Based Semantic Multi-View Localization](https://mlsanity.com/paper/1709.09905/) (2017)
- [Dogfight: Detecting Drones from Drones Videos](https://mlsanity.com/paper/2103.17242/) (2021)
- [YOLOPose: Transformer-based Multi-Object 6D Pose Estimation using Keypoint Regression](https://mlsanity.com/paper/2205.02536/) (2022)
- [Pose Estimation for Robot Manipulators via Keypoint Optimization and Sim-to-Real Transfer](https://mlsanity.com/paper/2010.08054/) (2020)
- [Real-Time Multi-View 3D Human Pose Estimation using Semantic Feedback to Smart Edge Sensors](https://mlsanity.com/paper/2106.14729/) (2021)
- [Stillleben: Realistic Scene Synthesis for Deep Learning in Robotics](https://mlsanity.com/paper/2005.05659/) (2020)
- [DeepURL: Deep Pose Estimation Framework for Underwater Relative Localization](https://mlsanity.com/paper/2003.05523/) (2020)
- [Online Marker-free Extrinsic Camera Calibration using Person Keypoint Detections](https://mlsanity.com/paper/2209.07393/) (2022)
- [3D Semantic Scene Perception using Distributed Smart Edge Sensors](https://mlsanity.com/paper/2205.01460/) (2022)
- [Underwater Multi-Robot Convoying using Visual Tracking by Detection](https://mlsanity.com/paper/1709.08292/) (2017)
- [6D Object Pose Estimation using Keypoints and Part Affinity Fields](https://mlsanity.com/paper/2107.02057/) (2021)
- [Real-time Pose Estimation from Images for Multiple Humanoid Robots](https://mlsanity.com/paper/2107.02675/) (2021)

## Built on by (most cited first, 4 of 4)

- [SLCF-Net: Sequential LiDAR-Camera Fusion for Semantic Scene Completion using a 3D Recurrent U-Net](https://mlsanity.com/paper/2403.08885/) (2024)
- [Real-time Holistic Robot Pose Estimation with Unknown States](https://mlsanity.com/paper/2402.05655/) (2024)
- [YOWO: You Only Walk Once to Jointly Map An Indoor Scene and Register Ceiling-mounted Cameras](https://mlsanity.com/paper/2511.16521/) (2025)
- [PoseDiff: A Unified Diffusion Model Bridging Robot Pose Estimation and Video-to-Action Control](https://mlsanity.com/paper/2509.24591/) (2025)

## Abstract (arXiv)

We present an approach for estimating a mobile robot's pose w.r.t. the allocentric coordinates of a network of static cameras using multi-view RGB images. The images are processed online, locally on smart edge sensors by deep neural networks to detect the robot and estimate 2D keypoints defined at distinctive positions of the 3D robot model. Robot keypoint detections are synchronized and fused on a central backend, where the robot's pose is estimated via multi-view minimization of reprojection errors. Through the pose estimation from external cameras, the robot's localization can be initialized in an allocentric map from a completely unknown state (kidnapped robot problem) and robustly tracked over time. We conduct a series of experiments evaluating the accuracy and robustness of the camera-based pose estimation compared to the robot's internal navigation stack, showing that our camera-based method achieves pose errors below 3 cm and 1deg and does not drift over time, as the robot is localized allocentrically. With the robot's pose precisely estimated, its observations can be fused into the allocentric scene model. We show a real-world application, where observations from mobile robot and static smart edge sensors are fused to collaboratively build a 3D semantic map of a ~240 m2 indoor environment.