mlsanityData to 7 Oct 2026

Gravity-Aware Monocular 3D Human-Object Reconstruction

arXiv 2108.08844 ↗First posted 19 Aug 2021cs.CVcited by 30

Not on the map. Why?

Abstract (arXiv)

This paper proposes GraviCap, i.e., a new approach for joint markerless 3D human motion capture and object trajectory estimation from monocular RGB videos. We focus on scenes with objects partially observed during a free flight. In contrast to existing monocular methods, we can recover scale, object trajectories as well as human bone lengths in meters and the ground plane's orientation, thanks to the awareness of the gravity constraining object motions. Our objective function is parametrised by the object's initial velocity and position, gravity direction and focal length, and jointly optimised for one or several free flight episodes. The proposed human-object interaction constraints ensure geometric consistency of the 3D reconstructions and improved physical plausibility of human poses compared to the unconstrained case. We evaluate GraviCap on a new dataset with ground-truth annotations for persons and different objects undergoing free flights. In the experiments, our approach achieves state-of-the-art accuracy in 3D human motion capture on various metrics. We urge the reader to watch our supplementary video. Both the source code and the dataset are released; see http://4dqv.mpi-inf.mpg.de/GraviCap/.

Key facts. Cited by 30 papers in the map (data through 7 Oct 2026). Builds on 36 of its 52 references that are in the map. First posted on arXiv 19 Aug 2021.

Builds on

36 of 52 references.

References · 36

  1. 2016Stacked Hourglass Networks for Human Pose Estimationcited by 913
  2. 2017End-to-end Recovery of Human Shape and Posecited by 883
  3. 2016Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Imagecited by 692
  4. 2019VIBE: Video Inference for Human Body Pose and Shape Estimationcited by 615
  5. 2016Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervisioncited by 493
  6. 2017A simple yet effective baseline for 3d human pose estimationcited by 401
  7. 20183D human pose estimation in video with temporal convolutions and semi-supervised trainingcited by 401
  8. 2016RMPE: Regional Multi-person Pose Estimationcited by 340
  9. 2019Resolving 3D Human Pose Ambiguities with 3D Scene Constraintscited by 294
  10. 2018Learning to Estimate 3D Human Pose and Shape from a Single Color Imagecited by 226
  11. 2016Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Posecited by 220
  12. 2020DeepCap: Monocular Human Performance Capture Using Weak Supervisioncited by 162
24 more
  1. 2020Perceiving 3D Human-Object Spatial Arrangements from a Single Image in the Wildcited by 152
  2. 2019Camera Distance-aware Top-down Approach for 3D Multi-person Pose Estimation from a Single RGB Imagecited by 131
  3. 2017VNect: Real-time 3D Human Pose Estimation with a Single RGB Cameracited by 129
  4. 20163D Human Pose Estimation = 2D Pose Estimation + Matchingcited by 112
  5. 2019Unsupervised 3D Pose Estimation with Geometric Self-Supervisioncited by 95
  6. 2020Contact and Human Dynamics from Monocular Videocited by 94
  7. 2018LCR-Net++: Multi-person 2D and 3D Pose Detection in Natural Imagescited by 88
  8. 2019RepNet: Weakly Supervised Training of an Adversarial Reprojection Network for 3D Human Pose Estimationcited by 85
  9. 20163D Human Pose Estimation from a Single Image via Distance Matrix Regressioncited by 73
  10. 2017Lifting from the Deep: Convolutional 3D Pose Estimation from a Single Imagecited by 73
  11. 2021Neural Monocular 3D Human Motion Capture with Physical Awarenesscited by 70
  12. 2019XNect: Real-time Multi-Person 3D Motion Capture with a Single RGB Cameracited by 68
  13. 2019EventCap: Monocular 3D Capture of High-Speed Human Motions using an Event Cameracited by 65
  14. 2019C3DPO: Canonical 3D Pose Networks for Non-Rigid Structure From Motioncited by 60
  15. 2020PhysCap: Physically Plausible Monocular 3D Motion Capture in Real Timecited by 50
  16. 2019Estimating 3D Motion and Forces of Person-Object Interactions from Monocular Videocited by 48
  17. 2019In the Wild Human Pose Estimation Using Explicit 2D Features and Intermediate 3D Representationscited by 48
  18. 2016Discriminative Correlation Filter with Channel and Spatial Reliabilitycited by 45
  19. 2016Structured Prediction of 3D Human Pose with Deep Neural Networkscited by 43
  20. 2020MotioNet: 3D Human Motion Reconstruction from Monocular Video with Skeleton Consistencycited by 31
  21. 2019Multi-Person 3D Human Pose Estimation from Monocular Imagescited by 19
  22. 20143D Shape Estimation from 2D Landmarks: A Convex Relaxation Approachcited by 11
  23. 2019Structure from Articulated Motion: Accurate and Stable Monocular 3D Reconstruction without Training Datacited by 5
  24. 2019Gravity as a Reference for Estimating a Person's Height from Videocited by 4

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 8; 2023: 9; 2024: 8; 2025: 4.

30 papers.

Sort:

By area:

  1. 2022BEHAVE: Dataset and Method for Tracking Human Object Interactionscited by 246
  2. 2021GLAMR: Global Occlusion-Aware Human Mesh Recovery with Dynamic Camerascited by 151
  3. 2022IMos: Intent-Driven Full-Body Motion Synthesis for Human-Object Interactionscited by 121
  4. 2022Human-Aware Object Placement for Visual Environment Reconstructioncited by 56
  5. 2024HOI-M3:Capture Multiple Humans and Objects Interaction within Contextual Environmentcited by 52
  6. 2024CORE4D: A 4D Human-Object-Human Interaction Dataset for Collaborative Object REarrangementcited by 40
  7. 2022D&D: Learning Human Dynamics from Dynamic Cameracited by 39
  8. 2023Object pop-up: Can we infer 3D objects and their poses from human interactions alone?cited by 36
  9. 2023PACE: Human and Camera Motion Estimation from in-the-wild Videoscited by 33
  10. 2022Scene-aware Egocentric 3D Human Pose Estimationcited by 30
  11. 2022HULC: 3D Human Motion Capture with Pose Manifold Sampling and Dense Contact Guidancecited by 29
  12. 2024Recent Trends in 3D Reconstruction of General Non-Rigid Scenescited by 28
18 more
  1. 2025A Survey on Human Interaction Motion Generationcited by 25
  2. 2024PACER+: On-Demand Pedestrian Animation Controller in Driving Scenarioscited by 24
  3. 2023ROAM: Robust and Object-Aware Motion Generation Using Neural Pose Descriptorscited by 23
  4. 2023Scene-Aware 3D Multi-Human Motion Capture from a Single Cameracited by 19
  5. 2022MoCapDeform: Monocular 3D Human Motion Capture in Deformable Scenescited by 18
  6. 2023Decaf: Monocular Deformation Capture for Face and Hand Interactionscited by 17
  7. 2022Discovering Human-Object Interaction Concepts via Self-Compositional Learningcited by 15
  8. 2024Contact-aware Human Motion Generation from Textual Descriptionscited by 15
  9. 2024COIN: Control-Inpainting Diffusion Prior for Human and Camera Motion Estimationcited by 15
  10. 2025Reconstructing 4D Spatial Intelligence: A Surveycited by 13
  11. 2023Compositional 3D Human-Object Neural Animationcited by 9
  12. 2024FORCE: Physics-aware Human-object Interactioncited by 7
  13. 20253D Human Interaction Generation: A Surveycited by 6
  14. 2023GraMMaR: Ground-aware Motion Model for 3D Human Motion Reconstructioncited by 4
  15. 2023Physically Plausible 3D Human-Scene Reconstruction from Monocular RGB Image using an Adversarial Learning Approachcited by 2
  16. 2023Utilizing Task-Generic Motion Prior to Recover Full-Body Motion from Very Sparse Signalscited by 2
  17. 2024TFS-NeRF: Template-Free NeRF for Semantic 3D Reconstruction of Dynamic Scenecited by 2
  18. 2025Physically Plausible Human-Object Rendering from Sparse Views via 3D Gaussian Splattingcited by 0