# MotioNet: 3D Human Motion Reconstruction from Monocular Video with Skeleton Consistency

arXiv: https://arxiv.org/abs/2006.12075 · HTML: https://mlsanity.com/paper/2006.12075/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Jun 2020.

## Built on by (most cited first, 30 of 31)

- [Human Motion Diffusion Model](https://mlsanity.com/paper/2209.14916/) (2022)
- [EDGE: Editable Dance Generation From Music](https://mlsanity.com/paper/2211.10658/) (2022)
- [A-NeRF: Articulated Neural Radiance Fields for Learning Human Shape, Appearance, and Pose](https://mlsanity.com/paper/2102.06199/) (2021)
- [World-Grounded Human Motion Recovery via Gravity-View Coordinates](https://mlsanity.com/paper/2409.06662/) (2024)
- [Avatars Grow Legs: Generating Smooth Human Motion from Sparse Tracking Inputs with Diffusion Model](https://mlsanity.com/paper/2304.08577/) (2023)
- [3D Human Pose Estimation via Intuitive Physics](https://mlsanity.com/paper/2303.18246/) (2023)
- [EMDM: Efficient Motion Diffusion Model for Fast and High-Quality Motion Generation](https://mlsanity.com/paper/2312.02256/) (2023)
- [Neural Monocular 3D Human Motion Capture with Physical Awareness](https://mlsanity.com/paper/2105.01057/) (2021)
- [MoDi: Unconditional Motion Synthesis from Diverse Data](https://mlsanity.com/paper/2206.08010/) (2022)
- [C·ASE: Learning Conditional Adversarial Skill Embeddings for Physics-based Characters](https://mlsanity.com/paper/2309.11351/) (2023)
- [DECO: Dense Estimation of 3D Human-Scene Contact In The Wild](https://mlsanity.com/paper/2309.15273/) (2023)
- [Conditional Directed Graph Convolution for 3D Human Pose Estimation](https://mlsanity.com/paper/2107.07797/) (2021)
- [Task-Generic Hierarchical Human Motion Prior using VAEs](https://mlsanity.com/paper/2106.04004/) (2021)
- [Gravity-Aware Monocular 3D Human-Object Reconstruction](https://mlsanity.com/paper/2108.08844/) (2021)
- [3D Human Motion Prediction: A Survey](https://mlsanity.com/paper/2203.01593/) (2022)
- [HULC: 3D Human Motion Capture with Pose Manifold Sampling and Dense Contact Guidance](https://mlsanity.com/paper/2205.05677/) (2022)
- [DiffDance: Cascaded Human Motion Diffusion Model for Dance Generation](https://mlsanity.com/paper/2308.02915/) (2023)
- [Adaptive Multi-view and Temporal Fusing Transformer for 3D Human Pose Estimation](https://mlsanity.com/paper/2110.05092/) (2021)
- [MoCapDeform: Monocular 3D Human Motion Capture in Deformable Scenes](https://mlsanity.com/paper/2208.08439/) (2022)
- [Unique Identification of 50,000+ Virtual Reality Users from Head & Hand Motion Data](https://mlsanity.com/paper/2302.08927/) (2023)
- [Example-based Motion Synthesis via Generative Motion Matching](https://mlsanity.com/paper/2306.00378/) (2023)
- [FLEX: Extrinsic Parameters-free Multi-view 3D Human Motion Reconstruction](https://mlsanity.com/paper/2105.01937/) (2021)
- [DH-AUG: DH Forward Kinematics Model Driven Augmentation for 3D Human Pose Estimation](https://mlsanity.com/paper/2207.09303/) (2022)
- [FusePose: IMU-Vision Sensor Fusion in Kinematic Space for Parametric Human Pose Estimation](https://mlsanity.com/paper/2208.11960/) (2022)
- [MMoFusion: Multi-modal Co-Speech Motion Generation with Diffusion Model](https://mlsanity.com/paper/2403.02905/) (2024)
- [KinTwin: Imitation Learning with Torque and Muscle Driven Biomechanical Models Enables Precise Replication of Able-Bodied and Impaired Movement from Markerless Motion Capture](https://mlsanity.com/paper/2505.13436/) (2025)
- [Pose Representations for Deep Skeletal Animation](https://mlsanity.com/paper/2111.13907/) (2021)
- [Skeleton-Graph: Long-Term 3D Motion Prediction From 2D Observations Using Deep Spatio-Temporal Graph CNNs](https://mlsanity.com/paper/2109.10257/) (2021)
- [Utilizing Task-Generic Motion Prior to Recover Full-Body Motion from Very Sparse Signals](https://mlsanity.com/paper/2308.15839/) (2023)
- [Motion Diffusion Autoencoders: Enabling Attribute Manipulation in Human Motion Demonstrated on Karate Techniques](https://mlsanity.com/paper/2501.18729/) (2025)

## Abstract (arXiv)

We introduce MotioNet, a deep neural network that directly reconstructs the motion of a 3D human skeleton from monocular video.While previous methods rely on either rigging or inverse kinematics (IK) to associate a consistent skeleton with temporally coherent joint rotations, our method is the first data-driven approach that directly outputs a kinematic skeleton, which is a complete, commonly used, motion representation. At the crux of our approach lies a deep neural network with embedded kinematic priors, which decomposes sequences of 2D joint positions into two separate attributes: a single, symmetric, skeleton, encoded by bone lengths, and a sequence of 3D joint rotations associated with global root positions and foot contact labels. These attributes are fed into an integrated forward kinematics (FK) layer that outputs 3D positions, which are compared to a ground truth. In addition, an adversarial loss is applied to the velocities of the recovered rotations, to ensure that they lie on the manifold of natural joint rotations. The key advantage of our approach is that it learns to infer natural joint rotations directly from the training data, rather than assuming an underlying model, or inferring them from joint positions using a data-agnostic IK solver. We show that enforcing a single consistent skeleton along with temporally coherent joint rotations constrains the solution space, leading to a more robust handling of self-occlusions and depth ambiguities.