# Semi-Perspective Decoupled Heatmaps for 3D Robot Pose Estimation from Depth Maps

arXiv: https://arxiv.org/abs/2207.02519 · HTML: https://mlsanity.com/paper/2207.02519/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 42 references that are in the map.
- First posted on arXiv 6 Jul 2022.

## Builds on (most cited first, 20 of 20)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size](https://mlsanity.com/paper/1602.07360/) (2016)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes](https://mlsanity.com/paper/1711.00199/) (2017)
- [A simple yet effective baseline for 3d human pose estimation](https://mlsanity.com/paper/1705.03098/) (2017)
- [Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomization](https://mlsanity.com/paper/1804.06516/) (2018)
- [Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Pose](https://mlsanity.com/paper/1611.07828/) (2016)
- [Deep Learning-Based Human Pose Estimation: A Survey](https://mlsanity.com/paper/2012.13392/) (2020)
- [Camera-to-Robot Pose Estimation from a Single Image](https://mlsanity.com/paper/1911.09231/) (2019)
- [Compressed Volumetric Heatmaps for Multi-Person 3D Pose Estimation](https://mlsanity.com/paper/2004.00329/) (2020)
- [Single-view robot pose and joint angle estimation via render & compare](https://mlsanity.com/paper/2104.09359/) (2021)
- [Kinect Range Sensing: Structured-Light versus Time-of-Flight Kinect](https://mlsanity.com/paper/1505.05459/) (2015)
- [Efficient Convolutional Neural Networks for Depth-Based Multi-Person Pose Estimation](https://mlsanity.com/paper/1912.00711/) (2019)
- [Indirect Object-to-Robot Pose Estimation from an External Monocular RGB Camera](https://mlsanity.com/paper/2008.11822/) (2020)
- [Real-time Convolutional Networks for Depth-based Human Pose Estimation](https://mlsanity.com/paper/1910.13911/) (2019)
- [Head Detection with Depth Images in the Wild](https://mlsanity.com/paper/1707.06786/) (2017)

## Built on by (most cited first, 6 of 6)

- [RoboPEPP: Vision-Based Robot Pose and Joint Angle Estimation through Embedding Predictive Pre-Training](https://mlsanity.com/paper/2411.17662/) (2024)
- [Bounding Box-Guided Diffusion for Synthesizing Industrial Images and Segmentation Map](https://mlsanity.com/paper/2505.03623/) (2025)
- [Depth-based Privileged Information for Boosting 3D Human Pose Estimation on RGB](https://mlsanity.com/paper/2409.11104/) (2024)
- [3D Pose Nowcasting: Forecast the Future to Improve the Present](https://mlsanity.com/paper/2308.12914/) (2023)
- [Latent Representations for Visual Proprioception in Inexpensive Robots](https://mlsanity.com/paper/2504.14634/) (2025)
- [PoseDiff: A Unified Diffusion Model Bridging Robot Pose Estimation and Video-to-Action Control](https://mlsanity.com/paper/2509.24591/) (2025)

## Abstract (arXiv)

Knowing the exact 3D location of workers and robots in a collaborative environment enables several real applications, such as the detection of unsafe situations or the study of mutual interactions for statistical and social purposes. In this paper, we propose a non-invasive and light-invariant framework based on depth devices and deep neural networks to estimate the 3D pose of robots from an external camera. The method can be applied to any robot without requiring hardware access to the internal states. We introduce a novel representation of the predicted pose, namely Semi-Perspective Decoupled Heatmaps (SPDH), to accurately compute 3D joint locations in world coordinates adapting efficient deep networks designed for the 2D Human Pose Estimation. The proposed approach, which takes as input a depth representation based on XYZ coordinates, can be trained on synthetic depth data and applied to real-world settings without the need for domain adaptation techniques. To this end, we present the SimBa dataset, based on both synthetic and real depth images, and use it for the experimental evaluation. Results show that the proposed approach, made of a specific depth map representation and the SPDH, overcomes the current state of the art.