# Dynamic Multi-Person Mesh Recovery From Uncalibrated Multi-View Cameras

arXiv: https://arxiv.org/abs/2110.10355 · HTML: https://mlsanity.com/paper/2110.10355/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 70 references that are in the map.
- First posted on arXiv 20 Oct 2021.

## Builds on (most cited first, 27 of 27)

- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [AMASS: Archive of Motion Capture as Surface Shapes](https://mlsanity.com/paper/1904.03278/) (2019)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loop](https://mlsanity.com/paper/1909.12828/) (2019)
- [VIBE: Video Inference for Human Body Pose and Shape Estimation](https://mlsanity.com/paper/1912.05656/) (2019)
- [RMPE: Regional Multi-person Pose Estimation](https://mlsanity.com/paper/1612.00137/) (2016)
- [Omni-Scale Feature Learning for Person Re-Identification](https://mlsanity.com/paper/1905.00953/) (2019)
- [Character Controllers Using Motion VAEs](https://mlsanity.com/paper/2103.14274/) (2021)
- [Panoptic Studio: A Massively Multiview System for Social Interaction Capture](https://mlsanity.com/paper/1612.03153/) (2016)
- [Coherent Reconstruction of Multiple Humans from a Single Image](https://mlsanity.com/paper/2006.08586/) (2020)
- [3D Human Motion Estimation via Motion Compression and Refinement](https://mlsanity.com/paper/2008.03789/) (2020)
- [Towards Accurate Markerless Human Shape and Pose Estimation over Time](https://mlsanity.com/paper/1707.07548/) (2017)
- [Fast and Robust Multi-Person 3D Pose Estimation from Multiple Views](https://mlsanity.com/paper/1901.04111/) (2019)
- [4D Association Graph for Realtime Multi-person Motion Capture Using Multiple Video Cameras](https://mlsanity.com/paper/2002.12625/) (2020)
- [Camera Calibration and Player Localization in SoccerNet-v2 and Investigation of their Representations for Action Spotting](https://mlsanity.com/paper/2104.09333/) (2021)
- [Multi-View Multi-Person 3D Pose Estimation with Plane Sweep Stereo](https://mlsanity.com/paper/2104.02273/) (2021)
- [MonoCap: Monocular Human Motion Capture using a CNN Coupled with a Geometric Prior](https://mlsanity.com/paper/1701.02354/) (2017)
- [Cross-View Tracking for Multi-Human 3D Pose Estimation at over 100 FPS](https://mlsanity.com/paper/2003.03972/) (2020)
- [Sports Camera Calibration via Synthetic Data](https://mlsanity.com/paper/1810.10658/) (2018)
- [General Dynamic Scene Reconstruction from Multiple View Video](https://mlsanity.com/paper/1509.09294/) (2015)
- [Real-Time Camera Pose Estimation for Sports Fields](https://mlsanity.com/paper/2003.14109/) (2020)
- [Part-Aware Measurement for Robust Multi-View Multi-Human 3D Pose Estimation and Tracking](https://mlsanity.com/paper/2106.11589/) (2021)
- [Recovering Trajectories of Unmarked Joints in 3D Human Actions Using Latent Space Optimization](https://mlsanity.com/paper/2012.02043/) (2020)
- [Camera Calibration from Dynamic Silhouettes Using Motion Barcodes](https://mlsanity.com/paper/1506.07866/) (2015)

## Built on by (most cited first, 26 of 26)

- [GLAMR: Global Occlusion-Aware Human Mesh Recovery with Dynamic Cameras](https://mlsanity.com/paper/2112.01524/) (2021)
- [PACE: Human and Camera Motion Estimation from in-the-wild Videos](https://mlsanity.com/paper/2310.13768/) (2023)
- [Neural MoCon: Neural Motion Control for Physically Plausible Human Motion Capture](https://mlsanity.com/paper/2203.14065/) (2022)
- [WHAC: World-grounded Humans and Cameras](https://mlsanity.com/paper/2403.12959/) (2024)
- [Reconstructing Groups of People with Hypergraph Relational Reasoning](https://mlsanity.com/paper/2308.15844/) (2023)
- [Occluded Human Body Capture with Self-Supervised Spatial-Temporal Motion Prior](https://mlsanity.com/paper/2207.05375/) (2022)
- [Rendering Humans from Object-Occluded Monocular Videos](https://mlsanity.com/paper/2308.04622/) (2023)
- [Learning Visibility Field for Detailed 3D Human Reconstruction and Relighting](https://mlsanity.com/paper/2304.11900/) (2023)
- [Deep learning for 3D human pose estimation and mesh recovery: A survey](https://mlsanity.com/paper/2402.18844/) (2024)
- [Delving Deep into Pixel Alignment Feature for Accurate Multi-view Human Mesh Recovery](https://mlsanity.com/paper/2301.06020/) (2023)
- [SmartMocap: Joint Estimation of Human and Camera Motion using Uncalibrated RGB Cameras](https://mlsanity.com/paper/2209.13906/) (2022)
- [Multi-view Human Body Mesh Translator](https://mlsanity.com/paper/2210.01886/) (2022)
- [Simultaneously Recovering Multi-Person Meshes and Multi-View Cameras with Human Semantics](https://mlsanity.com/paper/2412.18785/) (2024)
- [Spatiotemporal Multi-Camera Calibration using Freely Moving People](https://mlsanity.com/paper/2502.12546/) (2025)
- [NeMo: 3D Neural Motion Fields from Multiple Video Instances of the Same Action](https://mlsanity.com/paper/2212.13660/) (2022)
- [Physics-Guided Human Motion Capture with Pose Probability Modeling](https://mlsanity.com/paper/2308.09910/) (2023)
- [Multi-View Person Matching and 3D Pose Estimation with Arbitrary Uncalibrated Camera Networks](https://mlsanity.com/paper/2312.01561/) (2023)
- [SkelFormer: Markerless 3D Pose and Shape Estimation using Skeletal Transformers](https://mlsanity.com/paper/2404.12625/) (2024)
- [HeatFormer: A Neural Optimizer for Multiview Human Mesh Recovery](https://mlsanity.com/paper/2412.04456/) (2024)
- [MMHU: A Massive-Scale Multimodal Benchmark for Human Behavior Understanding](https://mlsanity.com/paper/2507.12463/) (2025)
- [RatBodyFormer: Rat Body Surface from Keypoints](https://mlsanity.com/paper/2412.09599/) (2024)
- [YOWO: You Only Walk Once to Jointly Map An Indoor Scene and Register Ceiling-mounted Cameras](https://mlsanity.com/paper/2511.16521/) (2025)
- [COMPOSE: Hypergraph Cover Optimization for Multi-view 3D Human Pose Estimation](https://mlsanity.com/paper/2601.09698/) (2026)
- [Unified Simulation, Perception, and Generation of Human Behavior](https://mlsanity.com/paper/2204.13678/) (2022)
- [MirrorCalib: Utilizing Human Pose Information for Mirror-based Virtual Camera Calibration](https://mlsanity.com/paper/2311.02791/) (2023)
- [BundleMoCap: Efficient, Robust and Smooth Motion Capture from Sparse Multiview Videos](https://mlsanity.com/paper/2311.12679/) (2023)

## Abstract (arXiv)

Dynamic multi-person mesh recovery has been a hot topic in 3D vision recently. However, few works focus on the multi-person motion capture from uncalibrated cameras, which mainly faces two challenges: the one is that inter-person interactions and occlusions introduce inherent ambiguities for both camera calibration and motion capture; The other is that a lack of dense correspondences can be used to constrain sparse camera geometries in a dynamic multi-person scene. Our key idea is incorporating motion prior knowledge into simultaneous optimization of extrinsic camera parameters and human meshes from noisy human semantics. First, we introduce a physics-geometry consistency to reduce the low and high frequency noises of the detected human semantics. Then a novel latent motion prior is proposed to simultaneously optimize extrinsic camera parameters and coherent human motions from slightly noisy inputs. Experimental results show that accurate camera parameters and human motions can be obtained through one-stage optimization. The codes will be publicly available at~urlhttps://www.yangangwang.com.