# Simultaneously Recovering Multi-Person Meshes and Multi-View Cameras with Human Semantics

arXiv: https://arxiv.org/abs/2412.18785 · HTML: https://mlsanity.com/paper/2412.18785/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 83 references that are in the map.
- First posted on arXiv 25 Dec 2024.

## Builds on (most cited first, 30 of 34)

- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [AMASS: Archive of Motion Capture as Surface Shapes](https://mlsanity.com/paper/1904.03278/) (2019)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image](https://mlsanity.com/paper/1607.08128/) (2016)
- [Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loop](https://mlsanity.com/paper/1909.12828/) (2019)
- [VIBE: Video Inference for Human Body Pose and Shape Estimation](https://mlsanity.com/paper/1912.05656/) (2019)
- [Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervision](https://mlsanity.com/paper/1611.09813/) (2016)
- [Omni-Scale Feature Learning for Person Re-Identification](https://mlsanity.com/paper/1905.00953/) (2019)
- [AlphaPose: Whole-Body Regional Multi-Person Pose Estimation and Tracking in Real-Time](https://mlsanity.com/paper/2211.03375/) (2022)
- [Character Controllers Using Motion VAEs](https://mlsanity.com/paper/2103.14274/) (2021)
- [Panoptic Studio: A Massively Multiview System for Social Interaction Capture](https://mlsanity.com/paper/1612.03153/) (2016)
- [Coherent Reconstruction of Multiple Humans from a Single Image](https://mlsanity.com/paper/2006.08586/) (2020)
- [3D Human Motion Estimation via Motion Compression and Refinement](https://mlsanity.com/paper/2008.03789/) (2020)
- [Towards Accurate Markerless Human Shape and Pose Estimation over Time](https://mlsanity.com/paper/1707.07548/) (2017)
- [Fast and Robust Multi-Person 3D Pose Estimation from Multiple Views](https://mlsanity.com/paper/1901.04111/) (2019)
- [Direct Multi-view Multi-person 3D Pose Estimation](https://mlsanity.com/paper/2111.04076/) (2021)
- [4D Association Graph for Realtime Multi-person Motion Capture Using Multiple Video Cameras](https://mlsanity.com/paper/2002.12625/) (2020)
- [Camera Calibration and Player Localization in SoccerNet-v2 and Investigation of their Representations for Action Spotting](https://mlsanity.com/paper/2104.09333/) (2021)
- [Multi-View Multi-Person 3D Pose Estimation with Plane Sweep Stereo](https://mlsanity.com/paper/2104.02273/) (2021)
- [Cross-View Tracking for Multi-Human 3D Pose Estimation at over 100 FPS](https://mlsanity.com/paper/2003.03972/) (2020)
- [Lightweight Multi-person Total Motion Capture Using Sparse Multi-view Cameras](https://mlsanity.com/paper/2108.10378/) (2021)
- [Shape-Aware Human Pose and Shape Reconstruction Using Multi-View Images](https://mlsanity.com/paper/1908.09464/) (2019)
- [Dynamic Multi-Person Mesh Recovery From Uncalibrated Multi-View Cameras](https://mlsanity.com/paper/2110.10355/) (2021)
- [Shape-aware Multi-Person Pose Estimation from Multi-View Images](https://mlsanity.com/paper/2110.02330/) (2021)
- [General Dynamic Scene Reconstruction from Multiple View Video](https://mlsanity.com/paper/1509.09294/) (2015)
- [Reconstructing Interacting Hands with Interaction Prior from Monocular Images](https://mlsanity.com/paper/2308.14082/) (2023)
- [An Integrated Platform for Live 3D Human Reconstruction and Motion Capturing](https://mlsanity.com/paper/1712.03084/) (2017)
- [Part-Aware Measurement for Robust Multi-View Multi-Human 3D Pose Estimation and Tracking](https://mlsanity.com/paper/2106.11589/) (2021)

## Built on by (most cited first, 3 of 3)

- [Point2Quad: Generating Quad Meshes from Point Clouds via Face Prediction](https://mlsanity.com/paper/2504.19545/) (2025)
- [DyCrowd: Towards Dynamic Crowd Reconstruction from a Large-scene Video](https://mlsanity.com/paper/2508.12644/) (2025)
- [Feed-Forward Multi-view Multi-person Reconstruction with Contrastive Human-Aware 3D Representation](https://mlsanity.com/paper/2609.00745/) (2026)

## Abstract (arXiv)

Dynamic multi-person mesh recovery has broad applications in sports broadcasting, virtual reality, and video games. However, current multi-view frameworks rely on a time-consuming camera calibration procedure. In this work, we focus on multi-person motion capture with uncalibrated cameras, which mainly faces two challenges: one is that inter-person interactions and occlusions introduce inherent ambiguities for both camera calibration and motion capture; the other is that a lack of dense correspondences can be used to constrain sparse camera geometries in a dynamic multi-person scene. Our key idea is to incorporate motion prior knowledge to simultaneously estimate camera parameters and human meshes from noisy human semantics. We first utilize human information from 2D images to initialize intrinsic and extrinsic parameters. Thus, the approach does not rely on any other calibration tools or background features. Then, a pose-geometry consistency is introduced to associate the detected humans from different views. Finally, a latent motion prior is proposed to refine the camera parameters and human motions. Experimental results show that accurate camera parameters and human motions can be obtained through a one-step reconstruction. The code are publicly available at~urlhttps://github.com/boycehbz/DMMR.