# Multi-View Multi-Person 3D Pose Estimation with Plane Sweep Stereo

arXiv: https://arxiv.org/abs/2104.02273 · HTML: https://mlsanity.com/paper/2104.02273/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 27 references that are in the map.
- First posted on arXiv 6 Apr 2021.

## Builds on (most cited first, 17 of 17)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [Integral Human Pose Regression](https://mlsanity.com/paper/1711.08229/) (2017)
- [Panoptic Studio: A Massively Multiview System for Social Interaction Capture](https://mlsanity.com/paper/1612.03153/) (2016)
- [Learnable Triangulation of Human Pose](https://mlsanity.com/paper/1905.05754/) (2019)
- [DPSNet: End-to-end Deep Plane Sweep Stereo](https://mlsanity.com/paper/1905.00538/) (2019)
- [Cross View Fusion for 3D Human Pose Estimation](https://mlsanity.com/paper/1909.01203/) (2019)
- [Fast and Robust Multi-Person 3D Pose Estimation from Multiple Views](https://mlsanity.com/paper/1901.04111/) (2019)
- [Camera Style Adaptation for Person Re-identification](https://mlsanity.com/paper/1711.10295/) (2017)
- [Weakly-Supervised 3D Human Pose Learning via Multi-view Images in the Wild](https://mlsanity.com/paper/2003.07581/) (2020)
- [Lightweight Multi-View 3D Pose Estimation through Camera-Disentangled Representation](https://mlsanity.com/paper/2004.02186/) (2020)
- [A generalizable approach for multi-view 3D human pose regression](https://mlsanity.com/paper/1804.10462/) (2018)
- [Multi-person 3D Pose Estimation in Crowded Scenes Based on Multi-View Geometry](https://mlsanity.com/paper/2007.10986/) (2020)
- [MetaFuse: A Pre-trained Fusion Model for Human Pose Estimation](https://mlsanity.com/paper/2003.13239/) (2020)

## Built on by (most cited first, 30 of 38)

- [DeepMultiCap: Performance Capture of Multiple Characters Using Sparse Multiview Cameras](https://mlsanity.com/paper/2105.00261/) (2021)
- [Direct Multi-view Multi-person 3D Pose Estimation](https://mlsanity.com/paper/2111.04076/) (2021)
- [Diffusion 3D Features (Diff3F): Decorating Untextured Shapes with Distilled Semantic Features](https://mlsanity.com/paper/2311.17024/) (2023)
- [Faster VoxelPose: Real-time 3D Human Pose Estimation by Orthographic Projection](https://mlsanity.com/paper/2207.10955/) (2022)
- [Graph-Based 3D Multi-Person Pose Estimation Using Multi-View Images](https://mlsanity.com/paper/2109.05885/) (2021)
- [Lightweight Multi-person Total Motion Capture Using Sparse Multi-view Cameras](https://mlsanity.com/paper/2108.10378/) (2021)
- [Dynamic Multi-Person Mesh Recovery From Uncalibrated Multi-View Cameras](https://mlsanity.com/paper/2110.10355/) (2021)
- [TEMPO: Efficient Multi-View Pose Estimation, Tracking, and Forecasting](https://mlsanity.com/paper/2309.07910/) (2023)
- [SelfPose3d: Self-Supervised Multi-Person Multi-View 3d Pose Estimation](https://mlsanity.com/paper/2404.02041/) (2024)
- [Multiple View Geometry Transformers for 3D Human Pose Estimation](https://mlsanity.com/paper/2311.10983/) (2023)
- [Proactive Multi-Camera Collaboration For 3D Human Pose Estimation](https://mlsanity.com/paper/2303.03767/) (2023)
- [Markerless human pose estimation for biomedical applications: a survey](https://mlsanity.com/paper/2308.00519/) (2023)
- [Human-M3: A Multi-view Multi-modal Dataset for 3D Human Pose Estimation in Outdoor Scenes](https://mlsanity.com/paper/2308.00628/) (2023)
- [POEM: Reconstructing Hand in a Point Embedded Multi-view Stereo](https://mlsanity.com/paper/2304.04038/) (2023)
- [Markerless Multi-view 3D Human Pose Estimation: a survey](https://mlsanity.com/paper/2407.03817/) (2024)
- [MV-SSM: Multi-View State Space Modeling for 3D Human Pose Estimation](https://mlsanity.com/paper/2509.00649/) (2025)
- [A Dual-Masked Auto-Encoder for Robust Motion Capture with Spatial-Temporal Skeletal Token Completion](https://mlsanity.com/paper/2207.07381/) (2022)
- [Reconstructing Close Human Interactions from Multiple Views](https://mlsanity.com/paper/2401.16173/) (2024)
- [AvatarPose: Avatar-guided 3D Pose Estimation of Close Human Interaction from Sparse Multi-view Videos](https://mlsanity.com/paper/2408.02110/) (2024)
- [FusePose: IMU-Vision Sensor Fusion in Kinematic Space for Parametric Human Pose Estimation](https://mlsanity.com/paper/2208.11960/) (2022)
- [Multi-View Pedestrian Occupancy Prediction with a Novel Synthetic Dataset](https://mlsanity.com/paper/2412.13569/) (2024)
- [Permutation-Invariant Relational Network for Multi-person 3D Pose Estimation](https://mlsanity.com/paper/2204.04913/) (2022)
- [VoxelKeypointFusion: Generalizable Multi-View Multi-Person Pose Estimation](https://mlsanity.com/paper/2410.18723/) (2024)
- [Simultaneously Recovering Multi-Person Meshes and Multi-View Cameras with Human Semantics](https://mlsanity.com/paper/2412.18785/) (2024)
- [Multi-person 3D pose estimation from unlabelled data](https://mlsanity.com/paper/2212.08731/) (2022)
- [Sports Analysis and VR Viewing System Based on Player Tracking and Pose Estimation with Multimodal and Multiview Sensors](https://mlsanity.com/paper/2405.01112/) (2024)
- [Every Angle Is Worth A Second Glance: Mining Kinematic Skeletal Structures from Multi-view Joint Cloud](https://mlsanity.com/paper/2502.02936/) (2025)
- [LiCamPose: Combining Multi-View LiDAR and RGB Cameras for Robust Single-timestamp 3D Human Pose Estimation](https://mlsanity.com/paper/2312.06409/) (2023)
- [Multi-person Physics-based Pose Estimation for Combat Sports](https://mlsanity.com/paper/2504.08175/) (2025)
- [COMPOSE: Hypergraph Cover Optimization for Multi-view 3D Human Pose Estimation](https://mlsanity.com/paper/2601.09698/) (2026)

## Abstract (arXiv)

Existing approaches for multi-view multi-person 3D pose estimation explicitly establish cross-view correspondences to group 2D pose detections from multiple camera views and solve for the 3D pose estimation for each person. Establishing cross-view correspondences is challenging in multi-person scenes, and incorrect correspondences will lead to sub-optimal performance for the multi-stage pipeline. In this work, we present our multi-view 3D pose estimation approach based on plane sweep stereo to jointly address the cross-view fusion and 3D pose reconstruction in a single shot. Specifically, we propose to perform depth regression for each joint of each 2D pose in a target camera view. Cross-view consistency constraints are implicitly enforced by multiple reference camera views via the plane sweep algorithm to facilitate accurate depth regression. We adopt a coarse-to-fine scheme to first regress the person-level depth followed by a per-person joint-level relative depth estimation. 3D poses are obtained from a simple back-projection given the estimated depths. We evaluate our approach on benchmark datasets where it outperforms previous state-of-the-arts while being remarkably efficient. Our code is available at https://github.com/jiahaoLjh/PlaneSweepPose.