# Towards Accurate Markerless Human Shape and Pose Estimation over Time

arXiv: https://arxiv.org/abs/1707.07548 · HTML: https://mlsanity.com/paper/1707.07548/

## Key facts

- Cited by 92 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jul 2017.

## Built on by (most cited first, 30 of 92)

- [HuMoR: 3D Human Motion Model for Robust Pose Estimation](https://mlsanity.com/paper/2105.04668/) (2021)
- [PyMAF: 3D Human Pose and Shape Regression with Pyramidal Mesh Alignment Feedback Loop](https://mlsanity.com/paper/2103.16507/) (2021)
- [BEHAVE: Dataset and Method for Tracking Human Object Interactions](https://mlsanity.com/paper/2204.06950/) (2022)
- [InterGen: Diffusion-based Multi-human Motion Generation under Complex Interactions](https://mlsanity.com/paper/2304.05684/) (2023)
- [PyMAF-X: Towards Well-aligned Full-body Model Regression from Monocular Images](https://mlsanity.com/paper/2207.06400/) (2022)
- [EgoBody: Human Body Shape and Motion of Interacting People from Head-Mounted Devices](https://mlsanity.com/paper/2112.07642/) (2021)
- [Collaborative Regression of Expressive Bodies using Moderation](https://mlsanity.com/paper/2105.05301/) (2021)
- [SimPoE: Simulated Character Control for 3D Human Pose Estimation](https://mlsanity.com/paper/2104.00683/) (2021)
- [TRAM: Global Trajectory and Motion of 3D Humans from in-the-wild Videos](https://mlsanity.com/paper/2403.17346/) (2024)
- [Recovering 3D Human Mesh from Monocular Images: A Survey](https://mlsanity.com/paper/2203.01923/) (2022)
- [Learning Motion Priors for 4D Human Body Capture in 3D Scenes](https://mlsanity.com/paper/2108.10399/) (2021)
- [InterCap: Joint Markerless 3D Tracking of Humans and Objects in Interaction](https://mlsanity.com/paper/2209.12354/) (2022)
- [Modeling Clothing as a Separate Layer for an Animatable Human Avatar](https://mlsanity.com/paper/2106.14879/) (2021)
- [Playing for 3D Human Recovery](https://mlsanity.com/paper/2110.07588/) (2021)
- [EgoHumans: An Egocentric 3D Multi-Human Benchmark](https://mlsanity.com/paper/2305.16487/) (2023)
- [DoubleField: Bridging the Neural Surface and Radiance Fields for High-fidelity Human Reconstruction and Rendering](https://mlsanity.com/paper/2106.03798/) (2021)
- [Accurate 3D Body Shape Regression using Metric and Semantic Attributes](https://mlsanity.com/paper/2206.07036/) (2022)
- [RoHM: Robust Human Motion Reconstruction via Diffusion](https://mlsanity.com/paper/2401.08570/) (2024)
- [ReFit: Recurrent Fitting Network for 3D Human Recovery](https://mlsanity.com/paper/2308.11184/) (2023)
- [LiDAR-aid Inertial Poser: Large-scale Human Motion Capture by Sparse Inertial and LiDAR Sensors](https://mlsanity.com/paper/2205.15410/) (2022)
- [3D Human Mesh Estimation from Virtual Markers](https://mlsanity.com/paper/2303.11726/) (2023)
- [Animal3D: A Comprehensive Dataset of 3D Animal Pose and Shape](https://mlsanity.com/paper/2308.11737/) (2023)
- [Probabilistic 3D Human Shape and Pose Estimation from Multiple Unconstrained Images in the Wild](https://mlsanity.com/paper/2103.10978/) (2021)
- [EmotionGesture: Audio-Driven Diverse Emotional Co-Speech 3D Gesture Generation](https://mlsanity.com/paper/2305.18891/) (2023)
- [Bilevel Online Adaptation for Out-of-Domain Human Mesh Reconstruction](https://mlsanity.com/paper/2103.16449/) (2021)
- [Dynamic Point Fields](https://mlsanity.com/paper/2304.02626/) (2023)
- [Task-Generic Hierarchical Human Motion Prior using VAEs](https://mlsanity.com/paper/2106.04004/) (2021)
- [mmBody Benchmark: 3D Body Reconstruction Dataset and Analysis for Millimeter Wave Radar](https://mlsanity.com/paper/2209.05070/) (2022)
- [Learning Parallel Dense Correspondence from Spatio-Temporal Descriptors for Efficient and Robust 4D Reconstruction](https://mlsanity.com/paper/2103.16341/) (2021)
- [Harmony4D: A Video Dataset for In-The-Wild Close Human Interactions](https://mlsanity.com/paper/2410.20294/) (2024)

## Abstract (arXiv)

Existing marker-less motion capture methods often assume known backgrounds, static cameras, and sequence specific motion priors, which narrows its application scenarios. Here we propose a fully automatic method that given multi-view video, estimates 3D human motion and body shape. We take recent SMPLify citebogo2016keep as the base method, and extend it in several ways. First we fit the body to 2D features detected in multi-view images. Second, we use a CNN method to segment the person in each image and fit the 3D body model to the contours to further improves accuracy. Third we utilize a generic and robust DCT temporal prior to handle the left and right side swapping issue sometimes introduced by the 2D pose estimator. Validation on standard benchmarks shows our results are comparable to the state of the art and also provide a realistic 3D shape avatar. We also demonstrate accurate results on HumanEva and on challenging dance sequences from YouTube in monocular case.