# MonoCap: Monocular Human Motion Capture using a CNN Coupled with a Geometric Prior

arXiv: https://arxiv.org/abs/1701.02354 · HTML: https://mlsanity.com/paper/1701.02354/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Jan 2017.

## Built on by (most cited first, 30 of 36)

- [Mesh Graphormer](https://mlsanity.com/paper/2104.00272/) (2021)
- [Kimera: from SLAM to Spatial Perception with 3D Dynamic Scene Graphs](https://mlsanity.com/paper/2101.06894/) (2021)
- [Cross-Attention of Disentangled Modalities for 3D Human Mesh Recovery with Transformers](https://mlsanity.com/paper/2207.13820/) (2022)
- [DANBO: Disentangled Articulated Neural Body Representations via Graph Neural Networks](https://mlsanity.com/paper/2205.01666/) (2022)
- [The One Where They Reconstructed 3D Humans and Environments in TV Shows](https://mlsanity.com/paper/2207.14279/) (2022)
- [Distribution-Aligned Diffusion for Human Mesh Recovery](https://mlsanity.com/paper/2308.13369/) (2023)
- [Dynamic Multi-Person Mesh Recovery From Uncalibrated Multi-View Cameras](https://mlsanity.com/paper/2110.10355/) (2021)
- [Sampling is Matter: Point-guided 3D Human Mesh Reconstruction](https://mlsanity.com/paper/2304.09502/) (2023)
- [Reconstructing Humans with a Biomechanically Accurate Skeleton](https://mlsanity.com/paper/2503.21751/) (2025)
- [BodyPressure -- Inferring Body Pose and Contact Pressure from a Depth Image](https://mlsanity.com/paper/2105.09936/) (2021)
- [Learning Visibility for Robust Dense Human Body Estimation](https://mlsanity.com/paper/2208.10652/) (2022)
- [StrobeNet: Category-Level Multiview Reconstruction of Articulated Objects](https://mlsanity.com/paper/2105.08016/) (2021)
- [MPT: Mesh Pre-Training with Transformers for Human Pose and Mesh Reconstruction](https://mlsanity.com/paper/2211.13357/) (2022)
- [Deep Monocular 3D Human Pose Estimation via Cascaded Dimension-Lifting](https://mlsanity.com/paper/2104.03520/) (2021)
- [A Methodological and Structural Review of Hand Gesture Recognition Across Diverse Data Modalities](https://mlsanity.com/paper/2408.05436/) (2024)
- [OmniPose: A Multi-Scale Framework for Multi-Person Pose Estimation](https://mlsanity.com/paper/2103.10180/) (2021)
- [3D Human Pose Estimation Using M"obius Graph Convolutional Networks](https://mlsanity.com/paper/2203.10554/) (2022)
- [Heuristic Weakly Supervised 3D Human Pose Estimation](https://mlsanity.com/paper/2105.10996/) (2021)
- [DC-GNet: Deep Mesh Relation Capturing Graph Convolution Network for 3D Human Shape Reconstruction](https://mlsanity.com/paper/2108.12384/) (2021)
- [BAPose: Bottom-Up Pose Estimation with Disentangled Waterfall Representations](https://mlsanity.com/paper/2112.10716/) (2021)
- [DropKey](https://mlsanity.com/paper/2208.02646/) (2022)
- [Multi-view Human Body Mesh Translator](https://mlsanity.com/paper/2210.01886/) (2022)
- [Occlusion Handling in 3D Human Pose Estimation with Perturbed Positional Encoding](https://mlsanity.com/paper/2405.17397/) (2024)
- [PONet: Robust 3D Human Pose Estimation via Learning Orientations Only](https://mlsanity.com/paper/2112.11153/) (2021)
- [DreamPose3D: Hallucinative Diffusion with Prompt Learning for 3D Human Pose Estimation](https://mlsanity.com/paper/2511.09502/) (2025)
- [On the role of depth predictions for 3D human pose estimation](https://mlsanity.com/paper/2103.02521/) (2021)
- [Adapted Human Pose: Monocular 3D Human Pose Estimation with Zero Real 3D Pose Data](https://mlsanity.com/paper/2105.10837/) (2021)
- [Can Action be Imitated? Learn to Reconstruct and Transfer Human Dynamics from Videos](https://mlsanity.com/paper/2107.11756/) (2021)
- [Joint 3D Human Shape Recovery and Pose Estimation from a Single Image with Bilayer Graph](https://mlsanity.com/paper/2110.08472/) (2021)
- [Parallel mesh reconstruction streams for pose estimation of interacting hands](https://mlsanity.com/paper/2104.12123/) (2021)

## Abstract (arXiv)

Recovering 3D full-body human pose is a challenging problem with many applications. It has been successfully addressed by motion capture systems with body worn markers and multiple cameras. In this paper, we address the more challenging case of not only using a single camera but also not leveraging markers: going directly from 2D appearance to 3D geometry. Deep learning approaches have shown remarkable abilities to discriminatively learn 2D appearance features. The missing piece is how to integrate 2D, 3D and temporal information to recover 3D geometry and account for the uncertainties arising from the discriminative model. We introduce a novel approach that treats 2D joint locations as latent variables whose uncertainty distributions are given by a deep fully convolutional neural network. The unknown 3D poses are modeled by a sparse representation and the 3D parameter estimates are realized via an Expectation-Maximization algorithm, where it is shown that the 2D joint location uncertainties can be conveniently marginalized out during inference. Extensive evaluation on benchmark datasets shows that the proposed approach achieves greater accuracy over state-of-the-art baselines. Notably, the proposed approach does not require synchronized 2D-3D data for training and is applicable to "in-the-wild" images, which is demonstrated with the MPII dataset.