# Real-Time Multi-View 3D Human Pose Estimation using Semantic Feedback to Smart Edge Sensors

arXiv: https://arxiv.org/abs/2106.14729 · HTML: https://mlsanity.com/paper/2106.14729/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 32 references that are in the map.
- First posted on arXiv 28 Jun 2021.

## Builds on (most cited first, 17 of 17)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [Simple Baselines for Human Pose Estimation and Tracking](https://mlsanity.com/paper/1804.06208/) (2018)
- [Cross View Fusion for 3D Human Pose Estimation](https://mlsanity.com/paper/1909.01203/) (2019)
- [Fast and Robust Multi-Person 3D Pose Estimation from Multiple Views](https://mlsanity.com/paper/1901.04111/) (2019)
- [Lightweight Multi-View 3D Pose Estimation through Camera-Disentangled Representation](https://mlsanity.com/paper/2004.02186/) (2020)
- [Harvesting Multiple Views for Marker-less 3D Human Pose Annotations](https://mlsanity.com/paper/1704.04793/) (2017)
- [An Evaluation of Edge TPU Accelerators for Convolutional Neural Networks](https://mlsanity.com/paper/2102.10423/) (2021)
- [Cross-View Tracking for Multi-Human 3D Pose Estimation at over 100 FPS](https://mlsanity.com/paper/2003.03972/) (2020)
- [Rethinking Pose in 3D: Multi-stage Refinement and Recovery for Markerless Motion Capture](https://mlsanity.com/paper/1808.01525/) (2018)
- [Iterative Greedy Matching for 3D Human Pose Tracking from Multiple Views](https://mlsanity.com/paper/2101.09745/) (2021)

## Built on by (most cited first, 16 of 16)

- [Faster VoxelPose: Real-time 3D Human Pose Estimation by Orthographic Projection](https://mlsanity.com/paper/2207.10955/) (2022)
- [Generalizable Human Pose Triangulation](https://mlsanity.com/paper/2110.00280/) (2021)
- [TEMPO: Efficient Multi-View Pose Estimation, Tracking, and Forecasting](https://mlsanity.com/paper/2309.07910/) (2023)
- [Towards Robust and Smooth 3D Multi-Person Pose Estimation from Monocular Videos in the Wild](https://mlsanity.com/paper/2309.08644/) (2023)
- [Probabilistic Triangulation for Uncalibrated Multi-View 3D Human Pose Estimation](https://mlsanity.com/paper/2309.04756/) (2023)
- [Online Marker-free Extrinsic Camera Calibration using Person Keypoint Detections](https://mlsanity.com/paper/2209.07393/) (2022)
- [3D Semantic Scene Perception using Distributed Smart Edge Sensors](https://mlsanity.com/paper/2205.01460/) (2022)
- [External Camera-based Mobile Robot Pose Estimation for Collaborative Perception with Smart Edge Sensors](https://mlsanity.com/paper/2303.03797/) (2023)
- [Deep Learning on Edge TPUs](https://mlsanity.com/paper/2108.13732/) (2021)
- [Object-level 3D Semantic Mapping using a Network of Smart Edge Sensors](https://mlsanity.com/paper/2211.11354/) (2022)
- [Semantic Prediction: Which One Should Come First, Recognition or Prediction?](https://mlsanity.com/paper/2110.02829/) (2021)
- [Cooperative Inference for Real-Time 3D Human Pose Estimation in Multi-Device Edge Networks](https://mlsanity.com/paper/2504.03052/) (2025)
- [YOWO: You Only Walk Once to Jointly Map An Indoor Scene and Register Ceiling-mounted Cameras](https://mlsanity.com/paper/2511.16521/) (2025)
- [Unconstrained Multi-view Human Pose Estimation with Algebraic Priors](https://mlsanity.com/paper/2604.24312/) (2026)
- [E3Pose: Energy-Efficient Edge-assisted Multi-camera System for Multi-human 3D Pose Estimation](https://mlsanity.com/paper/2301.09015/) (2023)
- [Marker-free Human Gait Analysis using a Smart Edge Sensor System](https://mlsanity.com/paper/2411.09538/) (2024)

## Abstract (arXiv)

We present a novel method for estimation of 3D human poses from a multi-camera setup, employing distributed smart edge sensors coupled with a backend through a semantic feedback loop. 2D joint detection for each camera view is performed locally on a dedicated embedded inference processor. Only the semantic skeleton representation is transmitted over the network and raw images remain on the sensor board. 3D poses are recovered from 2D joints on a central backend, based on triangulation and a body model which incorporates prior knowledge of the human skeleton. A feedback channel from backend to individual sensors is implemented on a semantic level. The allocentric 3D pose is backprojected into the sensor views where it is fused with 2D joint detections. The local semantic model on each sensor can thus be improved by incorporating global context information. The whole pipeline is capable of real-time operation. We evaluate our method on three public datasets, where we achieve state-of-the-art results and show the benefits of our feedback architecture, as well as in our own setup for multi-person experiments. Using the feedback signal improves the 2D joint detections and in turn the estimated 3D poses.