mlsanityData to 14 Oct 2026

World-Grounded Human Motion Recovery via Gravity-View Coordinates

arXiv 2409.06662 ↗First posted 10 Sep 2024cs.CV cs.AIcited by 137

Not on the map. Why?

Abstract (arXiv)

We present a novel method for recovering world-grounded human motion from monocular video. The main challenge lies in the ambiguity of defining the world coordinate system, which varies between sequences. Previous approaches attempt to alleviate this issue by predicting relative motion in an autoregressive manner, but are prone to accumulating errors. Instead, we propose estimating human poses in a novel Gravity-View (GV) coordinate system, which is defined by the world gravity and the camera view direction. The proposed GV system is naturally gravity-aligned and uniquely defined for each video frame, largely reducing the ambiguity of learning image-pose mapping. The estimated poses can be transformed back to the world coordinate system using camera rotations, forming a global motion sequence. Additionally, the per-frame estimation avoids error accumulation in the autoregressive methods. Experiments on in-the-wild benchmarks demonstrate that our method recovers more realistic motion in both the camera space and world-grounded settings, outperforming state-of-the-art methods in both accuracy and speed. The code is available at https://zju3dv.github.io/gvhmr/.

Key facts. Cited by 137 papers in the map (data through 14 Oct 2026). Builds on 33 of its 50 references that are in the map. First posted on arXiv 10 Sep 2024.

Builds on

33 of 50 references.

References · 33

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  3. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,612
  4. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,103
  5. 2022Human Motion Diffusion Modelcited by 1,080
  6. 2016Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Imagecited by 695
  7. 2021DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Camerascited by 687
  8. 2019Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loopcited by 627
  9. 2019VIBE: Video Inference for Human Body Pose and Shape Estimationcited by 620
  10. 2023Humans in 4D: Reconstructing and Tracking Humans with Transformerscited by 350
  11. 2021PARE: Part Attention Regressor for 3D Human Body Estimationcited by 337
  12. 2021HuMoR: 3D Human Motion Model for Robust Pose Estimationcited by 324
21 more
  1. 2018Learning 3D Human Dynamics from Videocited by 316
  2. 2022CLIFF: Carrying Location Information in Full Frames into Human Pose and Shape Estimationcited by 238
  3. 2023BEDLAM: A Synthetic Dataset of Bodies Exhibiting Detailed Lifelike Animated Motioncited by 236
  4. 2023WHAM: Reconstructing World-grounded Humans with Accurate 3D Motioncited by 206
  5. 2022Capturing and Inferring Dense Full-Body Human-Scene Contactcited by 180
  6. 2020Beyond Static Features for Temporally Consistent 3D Human Pose and Shape from a Videocited by 163
  7. 2021GLAMR: Global Occlusion-Aware Human Mesh Recovery with Dynamic Camerascited by 153
  8. 2022PyMAF-X: Towards Well-aligned Full-body Model Regression from Monocular Imagescited by 153
  9. 20203D Human Motion Estimation via Motion Compression and Refinementcited by 98
  10. 2023EMDB: The Electromagnetic Database of Global 3D Human Pose and Shape in the Wildcited by 97
  11. 2023TRACE: 5D Temporal Regression of Avatars with Dynamic Cameras in 3D Environmentscited by 91
  12. 2019Human Mesh Recovery from Monocular Images via a Skeleton-disentangled Representationcited by 87
  13. 2021Encoder-decoder with Multi-level Attention for 3D Human Shape and Pose Estimationcited by 62
  14. 2022Capturing Humans in Motion: Temporal-Attentive 3D Human Pose and Shape Estimation from Monocular Videocited by 60
  15. 2023Global-to-Local Modeling for Video-based 3D Human Pose and Shape Estimationcited by 47
  16. 2023ReFit: Recurrent Fitting Network for 3D Human Recoverycited by 47
  17. 2023NIKI: Neural Inverse Kinematics with Invertible Neural Networks for 3D Human Pose and Shape Estimationcited by 45
  18. 2022D&D: Learning Human Dynamics from Dynamic Cameracited by 41
  19. 2020MotioNet: 3D Human Motion Reconstruction from Monocular Video with Skeleton Consistencycited by 31
  20. 2024WHAC: World-grounded Humans and Camerascited by 30
  21. 2021TransPose: Real-time 3D Human Translation and Pose Estimation with Six Inertial Sensorscited by 25

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 46; 2026: 87.

137 papers.

Sort:

By area:

  1. 2025Uni3C: Unifying Precisely 3D-Enhanced Camera and Human Motion Controls for Video Generationcited by 77
  2. 2025PromptHMR: Promptable Human Mesh Recoverycited by 76
  3. 20243DTrajMaster: Mastering 3D Trajectory for Multi-Entity Motion in Video Generationcited by 71
  4. 2025Go to Zero: Towards Zero-shot Motion Generation with Million-scale Datacited by 68
  5. 2025KungfuBot: Physics-Based Humanoid Whole-Body Control for Learning Highly-Dynamic Skillscited by 52
  6. 2025GENMO: A GENeralist Model for Human MOtioncited by 49
  7. 2025Human3R: Everyone Everywhere All at Oncecited by 37
  8. 2025HY-Motion 1.0: Scaling Flow Matching Models for Text-To-Motion Generationcited by 36
  9. 2024Reconstructing People, Places, and Camerascited by 31
  10. 2025Joint Optimization for 4D Human-Scene Reconstruction in the Wildcited by 29
  11. 2025RealisDance-DiT: Simple yet Strong Baseline towards Controllable Character Animation in the Wildcited by 29
  12. 2025Ready-to-React: Online Reaction Policy for Two-Character Interaction Generationcited by 28
38 more
  1. 2025BEDLAM2.0: Synthetic Humans and Cameras in Motioncited by 25
  2. 2025Motion2Motion: Cross-topology Motion Transfer with Sparse Correspondencecited by 19
  3. 2025PhysHSI: Towards a Real-World Generalizable and Natural Humanoid-Scene Interaction Systemcited by 18
  4. 2025Precise Action-to-Video Generation Through Visual Action Promptscited by 15
  5. 2026HumanX: Toward Agile and Generalizable Humanoid Interaction Skills from Human Videoscited by 14
  6. 2026TextOp: Real-time Interactive Text-Driven Humanoid Robot Motion Generation and Controlcited by 14
  7. 2026Video Generation Models are General-Purpose Vision Learnerscited by 14
  8. 2025CoDA: Coordinated Diffusion Noise Optimization for Whole-Body Manipulation of Articulated Objectscited by 12
  9. 2025RealisMotion: Decomposed Human Motion Control and Video Generation in the World Spacecited by 12
  10. 2026LLaMo: Scaling Pretrained Language Models for Unified Motion Understanding and Generation with Continuous Autoregressive Tokenscited by 11
  11. 2026LPM 1.0: Video-based Character Performance Modelcited by 11
  12. 2025Humanoid Goalkeeper: Learning from Position Conditioned Task-Motion Constraintscited by 9
  13. 2026UniSH: Unifying Scene and Human Reconstruction in a Feed-Forward Passcited by 9
  14. 2025ODHSR: Online Dense 3D Reconstruction of Humans and Scenes from Monocular Videoscited by 8
  15. 2025Towards Adaptable Humanoid Control via Adaptive Motion Trackingcited by 8
  16. 2026WHOLE: World-Grounded Hand-Object Lifted from Egocentric Videoscited by 8
  17. 2025Target-Aware Video Diffusion Modelscited by 7
  18. 2025PhysHMR: Learning Humanoid Control Policies from Vision for Physically Plausible Human Motion Reconstructioncited by 7
  19. 2026DuoMo: Dual Motion Diffusion for World-Space Human Reconstructioncited by 7
  20. 2026ARGUS: Stacked Multi-View Identity Mosaic Injection for Subject-Preserving Video Generationcited by 7
  21. 2025HumanMM: Global Human Motion Recovery from Multi-shot Videoscited by 6
  22. 2025StableMotion: Training Motion Cleanup Models with Unpaired Corrupted Datacited by 6
  23. 2025HOI-PAGE: Zero-Shot Human-Object Interaction Generation with Part Affordance Guidancecited by 6
  24. 2026SOMA: Unifying Parametric Human Body Modelscited by 6
  25. 2024A Plug-and-Play Physical Motion Restoration Approach for In-the-Wild High-Difficulty Motionscited by 5
  26. 2025UniAct: Unified Motion Generation and Action Streaming for Humanoid Robotscited by 5
  27. 2026THFM: A Unified Video Foundation Model for 4D Human Perception and Beyondcited by 5
  28. 2025DAViD: Modeling Dynamic Affordance of 3D Objects Using Pre-trained Video Diffusion Modelscited by 4
  29. 2025MetricHMSR:Metric Human Mesh and Scene Recovery from Monocular Imagescited by 4
  30. 2025AdaViewPlanner: Adapting Video Diffusion Models for Viewpoint Planning in 4D Scenescited by 4
  31. 2026Towards Motion Turing Test: Evaluating Human-Likeness in Humanoid Robotscited by 4
  32. 2026OnlineHMR: Video-based Online World-Grounded Human Mesh Recoverycited by 4
  33. 2025ClimbingCap: Multi-Modal Dataset and Method for Rock Climbing in World Coordinatecited by 3
  34. 20253DV-TON: Textured 3D-Guided Consistent Video Try-on via Diffusion Modelscited by 3
  35. 2025Efficient and Scalable Monocular Human-Object Interaction Motion Reconstructioncited by 3
  36. 2026SafeFlow: Real-Time Text-Driven Humanoid Whole-Body Control via Physics-Guided Rectified Flow and Selective Safety Gatingcited by 3
  37. 2026Gloria: Consistent Character Video Generation via Content Anchorscited by 3
  38. 2026HumanScore: Benchmarking Human Motions in Generated Videoscited by 3

50 most cited of 137 All 137 (JSON)