# Multi-Person 3D Pose Estimation from Multi-View Uncalibrated Depth Cameras

arXiv: https://arxiv.org/abs/2401.15616 · HTML: https://mlsanity.com/paper/2401.15616/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 67 references that are in the map.
- First posted on arXiv 28 Jan 2024.

## Builds on (most cited first, 30 of 36)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [A simple yet effective baseline for 3d human pose estimation](https://mlsanity.com/paper/1705.03098/) (2017)
- [Recent Advances in Convolutional Neural Networks](https://mlsanity.com/paper/1512.07108/) (2015)
- [D2-Net: A Trainable CNN for Joint Detection and Description of Local Features](https://mlsanity.com/paper/1905.03561/) (2019)
- [LIFT: Learned Invariant Feature Transform](https://mlsanity.com/paper/1603.09114/) (2016)
- [HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimation](https://mlsanity.com/paper/1908.10357/) (2019)
- [Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Pose](https://mlsanity.com/paper/1611.07828/) (2016)
- [Single-Shot Multi-Person 3D Pose Estimation From Monocular RGB](https://mlsanity.com/paper/1712.03453/) (2017)
- [LF-Net: Learning Local Features from Images](https://mlsanity.com/paper/1805.09662/) (2018)
- [Camera Distance-aware Top-down Approach for 3D Multi-person Pose Estimation from a Single RGB Image](https://mlsanity.com/paper/1907.11346/) (2019)
- [3D Human Pose Estimation = 2D Pose Estimation + Matching](https://mlsanity.com/paper/1612.06524/) (2016)
- [Ordinal Depth Supervision for 3D Human Pose Estimation](https://mlsanity.com/paper/1805.04095/) (2018)
- [Single-Stage Multi-Person Pose Machines](https://mlsanity.com/paper/1908.09220/) (2019)
- [Generating Multiple Hypotheses for 3D Human Pose Estimation with Mixture Density Network](https://mlsanity.com/paper/1904.05547/) (2019)
- [LCR-Net++: Multi-person 2D and 3D Pose Detection in Natural Images](https://mlsanity.com/paper/1803.00455/) (2018)
- [Fast and Robust Multi-Person 3D Pose Estimation from Multiple Views](https://mlsanity.com/paper/1901.04111/) (2019)
- [Camera Relocalization by Computing Pairwise Relative Poses Using Convolutional Neural Network](https://mlsanity.com/paper/1707.09733/) (2017)
- [XNect: Real-time Multi-Person 3D Motion Capture with a Single RGB Camera](https://mlsanity.com/paper/1907.00837/) (2019)
- [Direct Multi-view Multi-person 3D Pose Estimation](https://mlsanity.com/paper/2111.04076/) (2021)
- [Rethinking on Multi-Stage Networks for Human Pose Estimation](https://mlsanity.com/paper/1901.00148/) (2019)
- [Relative Camera Pose Estimation Using Convolutional Neural Networks](https://mlsanity.com/paper/1702.01381/) (2017)
- [Graph-Based 3D Multi-Person Pose Estimation Using Multi-View Images](https://mlsanity.com/paper/2109.05885/) (2021)
- [Maximum-Margin Structured Learning with Deep Networks for 3D Human Pose Estimation](https://mlsanity.com/paper/1508.06708/) (2015)
- [Compressed Volumetric Heatmaps for Multi-Person 3D Pose Estimation](https://mlsanity.com/paper/2004.00329/) (2020)
- [Parameter-Efficient Person Re-identification in the 3D Space](https://mlsanity.com/paper/2006.04569/) (2020)

## Built on by (most cited first, 6 of 6)

- [Segment Anything Model for Road Network Graph Extraction](https://mlsanity.com/paper/2403.16051/) (2024)
- [Spatiotemporal Multi-Camera Calibration using Freely Moving People](https://mlsanity.com/paper/2502.12546/) (2025)
- [COMPOSE: Hypergraph Cover Optimization for Multi-view 3D Human Pose Estimation](https://mlsanity.com/paper/2601.09698/) (2026)
- [Unconstrained Multi-view Human Pose Estimation with Algebraic Priors](https://mlsanity.com/paper/2604.24312/) (2026)
- [HARMONI: Aligning Human and Scene Priors for Multi-View 4D Reconstruction](https://mlsanity.com/paper/2603.12789/) (2026)
- [Feed-Forward Multi-view Multi-person Reconstruction with Contrastive Human-Aware 3D Representation](https://mlsanity.com/paper/2609.00745/) (2026)

## Abstract (arXiv)

We tackle the task of multi-view, multi-person 3D human pose estimation from a limited number of uncalibrated depth cameras. Recently, many approaches have been proposed for 3D human pose estimation from multi-view RGB cameras. However, these works (1) assume the number of RGB camera views is large enough for 3D reconstruction, (2) the cameras are calibrated, and (3) rely on ground truth 3D poses for training their regression model. In this work, we propose to leverage sparse, uncalibrated depth cameras providing RGBD video streams for 3D human pose estimation. We present a simple pipeline for Multi-View Depth Human Pose Estimation (MVD-HPE) for jointly predicting the camera poses and 3D human poses without training a deep 3D human pose regression model. This framework utilizes 3D Re-ID appearance features from RGBD images to formulate more accurate correspondences (for deriving camera positions) compared to using RGB-only features. We further propose (1) depth-guided camera-pose estimation by leveraging 3D rigid transformations as guidance and (2) depth-constrained 3D human pose estimation by utilizing depth-projected 3D points as an alternative objective for optimization. In order to evaluate our proposed pipeline, we collect three video sets of RGBD videos recorded from multiple sparse-view depth cameras and ground truth 3D poses are manually annotated. Experiments show that our proposed method outperforms the current 3D human pose regression-free pipelines in terms of both camera pose estimation and 3D human pose estimation.