# SelfPose3d: Self-Supervised Multi-Person Multi-View 3d Pose Estimation

arXiv: https://arxiv.org/abs/2404.02041 · HTML: https://mlsanity.com/paper/2404.02041/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 50 of its 69 references that are in the map.
- First posted on arXiv 2 Apr 2024.

## Builds on (most cited first, 30 of 50)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [RandAugment: Practical automated data augmentation with a reduced search space](https://mlsanity.com/paper/1909.13719/) (2019)
- [Improved Regularization of Convolutional Neural Networks with Cutout](https://mlsanity.com/paper/1708.04552/) (2017)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image](https://mlsanity.com/paper/1607.08128/) (2016)
- [Simple Baselines for Human Pose Estimation and Tracking](https://mlsanity.com/paper/1804.06208/) (2018)
- [A simple yet effective baseline for 3d human pose estimation](https://mlsanity.com/paper/1705.03098/) (2017)
- [Cascaded Pyramid Network for Multi-Person Pose Estimation](https://mlsanity.com/paper/1711.07319/) (2017)
- [RMPE: Regional Multi-person Pose Estimation](https://mlsanity.com/paper/1612.00137/) (2016)
- [Integral Human Pose Regression](https://mlsanity.com/paper/1711.08229/) (2017)
- [HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimation](https://mlsanity.com/paper/1908.10357/) (2019)
- [Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderer](https://mlsanity.com/paper/1908.01210/) (2019)
- [Associative Embedding: End-to-End Learning for Joint Detection and Grouping](https://mlsanity.com/paper/1611.05424/) (2016)
- [Distribution-Aware Coordinate Representation for Human Pose Estimation](https://mlsanity.com/paper/1910.06278/) (2019)
- [Learnable Triangulation of Human Pose](https://mlsanity.com/paper/1905.05754/) (2019)
- [PifPaf: Composite Fields for Human Pose Estimation](https://mlsanity.com/paper/1903.06593/) (2019)
- [Epipolar Transformers](https://mlsanity.com/paper/2005.04551/) (2020)
- [Towards 3D Human Pose Estimation in the Wild: a Weakly-supervised Approach](https://mlsanity.com/paper/1704.02447/) (2017)
- [Unsupervised Learning of Object Landmarks through Conditional Image Generation](https://mlsanity.com/paper/1806.07823/) (2018)
- [LoopReg: Self-supervised Learning of Implicit Surface Correspondences, Pose and Shape for 3D Human Mesh Registration](https://mlsanity.com/paper/2010.12447/) (2020)
- [Self-Supervised Learning of 3D Human Pose using Multi-view Geometry](https://mlsanity.com/paper/1903.02330/) (2019)
- [Unsupervised 3D Pose Estimation with Geometric Self-Supervision](https://mlsanity.com/paper/1904.04812/) (2019)

## Built on by (most cited first, 21 of 21)

- [Beyond Role-Based Surgical Domain Modeling: Generalizable Re-Identification in the Operating Room](https://mlsanity.com/paper/2503.13028/) (2025)
- [DCHM: Depth-Consistent Human Modeling for Multiview Detection](https://mlsanity.com/paper/2507.14505/) (2025)
- [VoxelKeypointFusion: Generalizable Multi-View Multi-Person Pose Estimation](https://mlsanity.com/paper/2410.18723/) (2024)
- [Every Angle Is Worth A Second Glance: Mining Kinematic Skeletal Structures from Multi-view Joint Cloud](https://mlsanity.com/paper/2502.02936/) (2025)
- [RapidPoseTriangulation: Multi-view Multi-person Whole-body Human Pose Triangulation in a Millisecond](https://mlsanity.com/paper/2503.21692/) (2025)
- [TrackOR: Towards Personalized Intelligent Operating Rooms Through Robust Tracking](https://mlsanity.com/paper/2508.07968/) (2025)
- [An uncertainty-aware framework for data-efficient multi-view animal pose estimation](https://mlsanity.com/paper/2510.09903/) (2025)
- [RUMPL: Ray-Based Transformers for Universal Multi-View 2D to 3D Human Pose Lifting](https://mlsanity.com/paper/2512.15488/) (2025)
- [SimpleDepthPose: Fast and Reliable Human Pose Estimation with RGBD-Images](https://mlsanity.com/paper/2501.18478/) (2025)
- [GenHPE: Generative Counterfactuals for 3D Human Pose Estimation with Radio Frequency Signals](https://mlsanity.com/paper/2503.09537/) (2025)
- [COMPOSE: Hypergraph Cover Optimization for Multi-view 3D Human Pose Estimation](https://mlsanity.com/paper/2601.09698/) (2026)
- [Learning Context-Adaptive Motion Priors for Masked Motion Diffusion Models with Efficient Kinematic Attention Aggregation](https://mlsanity.com/paper/2603.07697/) (2026)
- [Unconstrained Multi-view Human Pose Estimation with Algebraic Priors](https://mlsanity.com/paper/2604.24312/) (2026)
- [X as Supervision: Contending with Depth Ambiguity in Unsupervised Monocular 3D Pose Estimation](https://mlsanity.com/paper/2411.13026/) (2024)
- [DeProPose: Deficiency-Proof 3D Human Pose Estimation via Adaptive Multi-View Fusion](https://mlsanity.com/paper/2502.16419/) (2025)
- [3D Ground Truth Reconstruction from Multi-Camera Annotations Using UKF](https://mlsanity.com/paper/2511.17609/) (2025)
- [Efficient Online 3D Multi-Camera Multi-Object Tracking and Pose Estimation](https://mlsanity.com/paper/2604.16522/) (2026)
- [Mesh-Aware Epipolar Matching for Multi-View Multi-Person 3D Pose Estimation in Basketball](https://mlsanity.com/paper/2605.29953/) (2026)
- [DisPOSE: Projected Polystochastic Diffusion for Self-Supervised Multi-View 3D Human Pose Estimation](https://mlsanity.com/paper/2606.07419/) (2026)
- [Multi-View In-Cabin Monitoring System for Public Transport Vehicles](https://mlsanity.com/paper/2606.11739/) (2026)
- [STA-TFM: Spatio-Temporal Aggregation Across Views TransForMer for Pose Estimation](https://mlsanity.com/paper/2609.24482/) (2026)

## Abstract (arXiv)

We present a new self-supervised approach, SelfPose3d, for estimating 3d poses of multiple persons from multiple camera views. Unlike current state-of-the-art fully-supervised methods, our approach does not require any 2d or 3d ground-truth poses and uses only the multi-view input images from a calibrated camera setup and 2d pseudo poses generated from an off-the-shelf 2d human pose estimator. We propose two self-supervised learning objectives: self-supervised person localization in 3d space and self-supervised 3d pose estimation. We achieve self-supervised 3d person localization by training the model on synthetically generated 3d points, serving as 3d person root positions, and on the projected root-heatmaps in all the views. We then model the 3d poses of all the localized persons with a bottleneck representation, map them onto all views obtaining 2d joints, and render them using 2d Gaussian heatmaps in an end-to-end differentiable manner. Afterwards, we use the corresponding 2d joints and heatmaps from the pseudo 2d poses for learning. To alleviate the intrinsic inaccuracy of the pseudo labels, we propose an adaptive supervision attention mechanism to guide the self-supervision. Our experiments and analysis on three public benchmark datasets, including Panoptic, Shelf, and Campus, show the effectiveness of our approach, which is comparable to fully-supervised methods. Code: https://github.com/CAMMA-public/SelfPose3D. Video demo: https://youtu.be/GAqhmUIr2E8.