# General Automatic Human Shape and Motion Capture Using Volumetric Contour Cues

arXiv: https://arxiv.org/abs/1607.08659 · HTML: https://mlsanity.com/paper/1607.08659/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Jul 2016.

## Built on by (most cited first, 15 of 15)

- [A-NeRF: Articulated Neural Radiance Fields for Learning Human Shape, Appearance, and Pose](https://mlsanity.com/paper/2102.06199/) (2021)
- [InterCap: Joint Markerless 3D Tracking of Humans and Objects in Interaction](https://mlsanity.com/paper/2209.12354/) (2022)
- [Registering Explicit to Implicit: Towards High-Fidelity Garment mesh Reconstruction from Single Images](https://mlsanity.com/paper/2203.15007/) (2022)
- [LayGA: Layered Gaussian Avatars for Animatable Clothing Transfer](https://mlsanity.com/paper/2405.07319/) (2024)
- [Detailed Avatar Recovery from Single Image](https://mlsanity.com/paper/2108.02931/) (2021)
- [Gaussian Shadow Casting for Neural Characters](https://mlsanity.com/paper/2401.06116/) (2024)
- [A Hybrid ANN-SNN Architecture for Low-Power and Low-Latency Visual Perception](https://mlsanity.com/paper/2303.14176/) (2023)
- [Self-supervised 3D Representation Learning of Dressed Humans from Social Media Videos](https://mlsanity.com/paper/2103.03319/) (2021)
- [HiFECap: Monocular High-Fidelity and Expressive Capture of Human Performances](https://mlsanity.com/paper/2210.05665/) (2022)
- [Grasp in Gaussians: Fast Monocular Reconstruction of Dynamic Hand-Object Interactions](https://mlsanity.com/paper/2604.12929/) (2026)
- [Deep unsupervised 3D human body reconstruction from a sparse set of landmarks](https://mlsanity.com/paper/2106.12282/) (2021)
- [3D Human Mesh Construction Leveraging Wi-Fi](https://mlsanity.com/paper/2210.10957/) (2022)
- [A Deeper Look into DeepCap](https://mlsanity.com/paper/2111.10563/) (2021)
- [Markerless Body Motion Capturing for 3D Character Animation based on Multi-view Cameras](https://mlsanity.com/paper/2212.05788/) (2022)
- [Representing Animatable Avatar via Factorized Neural Fields](https://mlsanity.com/paper/2406.00637/) (2024)

## Abstract (arXiv)

Markerless motion capture algorithms require a 3D body with properly personalized skeleton dimension and/or body shape and appearance to successfully track a person. Unfortunately, many tracking methods consider model personalization a different problem and use manual or semi-automatic model initialization, which greatly reduces applicability. In this paper, we propose a fully automatic algorithm that jointly creates a rigged actor model commonly used for animation - skeleton, volumetric shape, appearance, and optionally a body surface - and estimates the actor's motion from multi-view video input only. The approach is rigorously designed to work on footage of general outdoor scenes recorded with very few cameras and without background subtraction. Our method uses a new image formation model with analytic visibility and analytically differentiable alignment energy. For reconstruction, 3D body shape is approximated as Gaussian density field. For pose and shape estimation, we minimize a new edge-based alignment energy inspired by volume raycasting in an absorbing medium. We further propose a new statistical human body model that represents the body surface, volumetric Gaussian density, as well as variability in skeleton shape. Given any multi-view sequence, our method jointly optimizes the pose and shape parameters of this model fully automatically in a spatiotemporal way.