mlsanityData to 7 Oct 2026

Differentiable Dynamics for Articulated 3d Human Motion Reconstruction

arXiv 2205.12256 ↗First posted 24 May 2022cs.CVcited by 38

Not on the map. Why?

Abstract (arXiv)

We introduce DiffPhy, a differentiable physics-based model for articulated 3d human motion reconstruction from video. Applications of physics-based reasoning in human motion analysis have so far been limited, both by the complexity of constructing adequate physical models of articulated human motion, and by the formidable challenges of performing stable and efficient inference with physics in the loop. We jointly address such modeling and inference challenges by proposing an approach that combines a physically plausible body representation with anatomical joint limits, a differentiable physics simulator, and optimization techniques that ensure good performance and robustness to suboptimal local optima. In contrast to several recent methods, our approach readily supports full-body contact including interactions with objects in the scene. Most importantly, our model connects end-to-end with images, thus supporting direct gradient-based physics optimization by means of image-based loss functions. We validate the model by demonstrating that it can accurately reconstruct physically plausible 3d human motion from monocular video, both on public benchmarks with available 3d ground-truth, and on videos from the internet.

Key facts. Cited by 38 papers in the map (data through 7 Oct 2026). Builds on 30 of its 62 references that are in the map. First posted on arXiv 24 May 2022.

Builds on

30 of 62 references.

References · 30

  1. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,208
  2. 2017End-to-end Recovery of Human Shape and Posecited by 883
  3. 2019VIBE: Video Inference for Human Body Pose and Shape Estimationcited by 615
  4. 2021Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulationcited by 375
  5. 2021HuMoR: 3D Human Motion Model for Robust Pose Estimationcited by 319
  6. 2019DiffTaichi: Differentiable Programming for Physical Simulationcited by 248
  7. 2022BEHAVE: Dataset and Method for Tracking Human Object Interactionscited by 246
  8. 2018Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodiescited by 239
  9. 2017Single-Shot Multi-Person 3D Pose Estimation From Monocular RGBcited by 195
  10. 2020Long-term Human Motion Prediction with Scene Contextcited by 191
  11. 2018Monocular Total Capture: Posing Face, Body, and Hands in the Wildcited by 162
  12. 2018DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skillscited by 153
18 more
  1. 2020Perceiving 3D Human-Object Spatial Arrangements from a Single Image in the Wildcited by 152
  2. 2017VNect: Real-time 3D Human Pose Estimation with a Single RGB Cameracited by 129
  3. 2021SimPoE: Simulated Character Control for 3D Human Pose Estimationcited by 111
  4. 2021Dynamics-Regulated Kinematic Policy for Egocentric Pose Estimationcited by 101
  5. 2020Contact and Human Dynamics from Monocular Videocited by 94
  6. 2017Exploiting temporal information for 3D pose estimationcited by 89
  7. 2021Gradients are Not All You Needcited by 82
  8. 2020NeuralSim: Augmenting Differentiable Simulators with Neural Networkscited by 77
  9. 2021Neural Monocular 3D Human Motion Capture with Physical Awarenesscited by 70
  10. 2021Physics-based Human Motion Estimation and Synthesis from Videoscited by 65
  11. 2020Residual Force Control for Agile Human Behavior Imitation and Extended Motion Synthesiscited by 63
  12. 2020Neural Descent for Visual 3D Human Pose and Shapecited by 60
  13. 2020PhysCap: Physically Plausible Monocular 3D Motion Capture in Real Timecited by 50
  14. 2021Fast and Feature-Complete Differentiable Physics for Articulated Rigid Bodies with Contactcited by 50
  15. 2021THUNDR: Transformer-based 3D HUmaN Reconstruction with Markerscited by 50
  16. 2015Building Statistical Shape Spaces for 3D Human Modelingcited by 47
  17. 2021Efficient Differentiable Simulation of Articulated Bodiescited by 38
  18. 2021TransPose: Real-time 3D Human Translation and Pose Estimation with Six Inertial Sensorscited by 24

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 10; 2024: 13; 2025: 6; 2026: 8.

38 papers.

Sort:

By area:

  1. 2023EMAGE: Towards Unified Holistic Co-Speech Gesture Generation via Expressive Masked Audio Gesture Modelingcited by 101
  2. 20233D Human Pose Estimation via Intuitive Physicscited by 100
  3. 2023DECO: Dense Estimation of 3D Human-Scene Contact In The Wildcited by 54
  4. 2024RoHM: Robust Human Motion Reconstruction via Diffusioncited by 46
  5. 2024AnySkill: Learning Open-Vocabulary Physical Skill for Interactive Agentscited by 32
  6. 2023Physics-Informed Computer Vision: A Review and Perspectivescited by 28
  7. 2024PhysPT: Physics-aware Pretrained Transformer for Estimating Human Dynamics from Monocular Videoscited by 27
  8. 2024MotionCraft: Crafting Whole-Body Motion with Plug-and-Play Multimodal Controlscited by 24
  9. 2024MultiPhys: Multi-Person Physics-aware 3D Motion Estimationcited by 18
  10. 2023Decaf: Monocular Deformation Capture for Face and Hand Interactionscited by 17
  11. 2024Contact-aware Human Motion Generation from Textual Descriptionscited by 15
  12. 2022Transformer-Based Learned Optimizationcited by 13
26 more
  1. 2024ImDy: Human Inverse Dynamics from Imitated Observationscited by 9
  2. 2025FinePhys: Fine-grained Human Action Generation by Explicitly Incorporating Physical Laws for Effective Skeletal Guidancecited by 9
  3. 2023DeepSimHO: Stable Pose Estimation for Hand-Object Interaction via Physics Simulationcited by 8
  4. 2024Optimal-state Dynamics Estimation for Physics-based Human Motion Capture from Videoscited by 6
  5. 2024Learned Neural Physics Simulation for Articulated 3D Human Pose Reconstructioncited by 6
  6. 2024Human Motion Prediction under Unexpected Perturbationcited by 5
  7. 2025PhysHMR: Learning Humanoid Control Policies from Vision for Physically Plausible Human Motion Reconstructioncited by 5
  8. 2023GraMMaR: Ground-aware Motion Model for 3D Human Motion Reconstructioncited by 4
  9. 2024A Plug-and-Play Physical Motion Restoration Approach for In-the-Wild High-Difficulty Motionscited by 4
  10. 2023HandyPriors: Physically Consistent Perception of Hand-Object Interactions with Differentiable Priorscited by 3
  11. 2026QuaMo: Quaternion Motions for Vision-based 3D Human Kinematics Capturecited by 3
  12. 2023Physics-Guided Human Motion Capture with Pose Probability Modelingcited by 2
  13. 2024Bayesian Differentiable Physics for Cloth Digitalizationcited by 2
  14. 2025Prof. Robot: Differentiable Robot Rendering Without Static and Self-Collisionscited by 2
  15. 2024Homogeneous Dynamics Space for Heterogeneous Humanscited by 1
  16. 2025Multi-person Physics-based Pose Estimation for Combat Sportscited by 1
  17. 2025Beyond Motion Pattern: An Empirical Study of Physical Forces for Human Motion Understandingcited by 1
  18. 2026GMT: Goal-Conditioned Multimodal Transformer for 6-DOF Object Trajectory Synthesis in 3D Scenescited by 1
  19. 2026InterPhys: Physics-aware Human Motion Synthesis in a Dynamic Scenecited by 1
  20. 2023Semantic and Articulated Pedestrian Sensing Onboard a Moving Vehiclecited by 0
  21. 2025Improving Human Motion Plausibility with Body Momentumcited by 0
  22. 2026Electrostatics-Inspired Surface Reconstruction (EISR): Recovering 3D Shapes as a Superposition of Poisson's PDE Solutionscited by 0
  23. 2026LEGO: Latent-space Exploration for Geometry-aware Optimization of Humanoid Kinematic Designcited by 0
  24. 2026Discriminative-Generative Synergy for Occlusion Robust 3D Human Mesh Recoverycited by 0
  25. 2026Gravity-guided Contact Dynamics Estimation from 3D Human Motionscited by 0
  26. 2026Generalist Open-World Temporal Perceptioncited by 0