mlsanityData to 14 Oct 2026

WATCH: World-aware Allied Trajectory and pose reConstruction for Camera and Human

arXiv 2509.04600 ↗First posted 4 Sep 2025cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Reconstructing global human motion from monocular video is fundamental to VR, graphics, and robotics, yet remains ill-posed due to depth ambiguity, motion ambiguity, and the entanglement of camera and human movements. Human-motion-centric methods achieve strong physical plausibility but leave two signals unused: camera orientation is processed through a fixed coordinate transformation with no independent supervision of its components, and camera velocity is discarded entirely despite being directly observable from SLAM. Camera-trajectory-centric methods use camera translation directly, but hard-decoding SLAM trajectories into human positions propagates depth errors and fails entirely under static cameras. We present WATCH (World-aware Allied Trajectory and pose reConstruction for Camera and Human). The key observation is that once camera orientation is made explicit, camera velocity becomes a natural additional input rather than an ambiguous one. We therefore decompose camera rotation into a network-estimated roll-pitch component and an analytically recoverable yaw, supervising each independently. This decomposition exposes a clean geometric interface through which camera velocity is incorporated as a learned spatial prior in the backbone, without the physically implausible artifacts that arise from hard-decoding. WATCH outperforms prior human-motion-centric methods on both static-camera (RICH) and dynamic-camera (EMDB) benchmarks in global trajectory accuracy, temporal smoothness, and physical plausibility, and remains robust when ground-truth camera is replaced with DPVO estimates.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 51 of its 59 references that are in the map. First posted on arXiv 4 Sep 2025.

Builds on

51 of 59 references.

References · 51

  1. 2023Segment Anythingcited by 8,234
  2. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  3. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,612
  4. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,567
  5. 2025VGGT: Visual Geometry Grounded Transformercited by 1,539
  6. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,236
  7. 2017End-to-end Recovery of Human Shape and Posecited by 885
  8. 2016ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Camerascited by 806
  9. 2016Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Imagecited by 695
  10. 2021DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Camerascited by 687
  11. 2023ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depthcited by 657
  12. 2019Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loopcited by 627
38 more
  1. 2019VIBE: Video Inference for Human Body Pose and Shape Estimationcited by 620
  2. 2023Humans in 4D: Reconstructing and Tracking Humans with Transformerscited by 350
  3. 2021PARE: Part Attention Regressor for 3D Human Body Estimationcited by 337
  4. 2020HybrIK: A Hybrid Analytical-Neural Inverse Kinematics Solution for 3D Human Pose and Shape Estimationcited by 307
  5. 2021PyMAF: 3D Human Pose and Shape Regression with Pyramidal Mesh Alignment Feedback Loopcited by 250
  6. 2022CLIFF: Carrying Location Information in Full Frames into Human Pose and Shape Estimationcited by 238
  7. 2023BEDLAM: A Synthetic Dataset of Bodies Exhibiting Detailed Lifelike Animated Motioncited by 236
  8. 2023WHAM: Reconstructing World-grounded Humans with Accurate 3D Motioncited by 206
  9. 2022Deep Patch Visual Odometrycited by 190
  10. 2024HumanPlus: Humanoid Shadowing and Imitation from Humanscited by 185
  11. 2022Capturing and Inferring Dense Full-Body Human-Scene Contactcited by 180
  12. 2020Beyond Static Features for Temporally Consistent 3D Human Pose and Shape from a Videocited by 163
  13. 2021GLAMR: Global Occlusion-Aware Human Mesh Recovery with Dynamic Camerascited by 153
  14. 2023Decoupling Human and Camera Motion from Videos in the Wildcited by 142
  15. 2024World-Grounded Human Motion Recovery via Gravity-View Coordinatescited by 137
  16. 2024TRAM: Global Trajectory and Motion of 3D Humans from in-the-wild Videoscited by 106
  17. 2024Multi-HMR: Multi-Person Whole-Body Human Mesh Recovery in a Single Shotcited by 101
  18. 20203D Human Motion Estimation via Motion Compression and Refinementcited by 98
  19. 2025Controllable Video Generation: A Surveycited by 98
  20. 2023EMDB: The Electromagnetic Database of Global 3D Human Pose and Shape in the Wildcited by 97
  21. 2024TokenHMR: Advancing Human Mesh Recovery with a Tokenized Pose Representationcited by 95
  22. 2023TRACE: 5D Temporal Regression of Avatars with Dynamic Cameras in 3D Environmentscited by 91
  23. 2025PromptHMR: Promptable Human Mesh Recoverycited by 76
  24. 2021Physics-based Human Motion Estimation and Synthesis from Videoscited by 66
  25. 2024CameraHMR: Aligning People with Perspectivecited by 65
  26. 2021Encoder-decoder with Multi-level Attention for 3D Human Shape and Pose Estimationcited by 62
  27. 2022Capturing Humans in Motion: Temporal-Attentive 3D Human Pose and Shape Estimation from Monocular Videocited by 60
  28. 2025GENMO: A GENeralist Model for Human MOtioncited by 49
  29. 2023Global-to-Local Modeling for Video-based 3D Human Pose and Shape Estimationcited by 47
  30. 2023ReFit: Recurrent Fitting Network for 3D Human Recoverycited by 47
  31. 2022D&D: Learning Human Dynamics from Dynamic Cameracited by 41
  32. 2023PACE: Human and Camera Motion Estimation from in-the-wild Videoscited by 34
  33. 2024WHAC: World-grounded Humans and Camerascited by 30
  34. 2025Joint Optimization for 4D Human-Scene Reconstruction in the Wildcited by 29
  35. 2024PhysPT: Physics-aware Pretrained Transformer for Estimating Human Dynamics from Monocular Videoscited by 28
  36. 2024Synergistic Global-space Camera and Human Reconstruction from Videoscited by 17
  37. 2024AToM: Aligning Text-to-Motion Model at Event-Level with GPT-4Vision Rewardcited by 7
  38. 2025Generating Physically Realistic and Directable Human Motions from Multi-Modal Inputscited by 6

50 most cited of 51 All 51 (JSON)

Built on by

Not cited yet.