mlsanityData to 7 Oct 2026

HULC: 3D Human Motion Capture with Pose Manifold Sampling and Dense Contact Guidance

arXiv 2205.05677 ↗First posted 11 May 2022cs.CV cs.GR cs.HCcited by 29

Not on the map. Why?

Abstract (arXiv)

Marker-less monocular 3D human motion capture (MoCap) with scene interactions is a challenging research topic relevant for extended reality, robotics and virtual avatar generation. Due to the inherent depth ambiguity of monocular settings, 3D motions captured with existing methods often contain severe artefacts such as incorrect body-scene inter-penetrations, jitter and body floating. To tackle these issues, we propose HULC, a new approach for 3D human MoCap which is aware of the scene geometry. HULC estimates 3D poses and dense body-environment surface contacts for improved 3D localisations, as well as the absolute scale of the subject. Furthermore, we introduce a 3D pose trajectory optimisation based on a novel pose manifold sampling that resolves erroneous body-environment inter-penetrations. Although the proposed method requires less structured inputs compared to existing scene-aware monocular MoCap algorithms, it produces more physically-plausible poses: HULC significantly and consistently outperforms the existing approaches in various experiments and on different metrics. Project page: https://vcai.mpi-inf.mpg.de/projects/HULC/.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Builds on 53 of its 70 references that are in the map. First posted on arXiv 11 May 2022.

Builds on

53 of 70 references.

References · 53

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2013Auto-Encoding Variational Bayescited by 3,980
  6. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,600
  7. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,208
  8. 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,161
  9. 2016Stacked Hourglass Networks for Human Pose Estimationcited by 913
  10. 2017End-to-end Recovery of Human Shape and Posecited by 883
  11. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  12. 2016Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Imagecited by 692
38 more
  1. 2019Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loopcited by 625
  2. 2019VIBE: Video Inference for Human Body Pose and Shape Estimationcited by 615
  3. 2016Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervisioncited by 493
  4. 2017A simple yet effective baseline for 3d human pose estimationcited by 401
  5. 20183D human pose estimation in video with temporal convolutions and semi-supervised trainingcited by 401
  6. 2021PARE: Part Attention Regressor for 3D Human Body Estimationcited by 334
  7. 2021HuMoR: 3D Human Motion Model for Robust Pose Estimationcited by 319
  8. 2018Learning 3D Human Dynamics from Videocited by 315
  9. 2019Resolving 3D Human Pose Ambiguities with 3D Scene Constraintscited by 294
  10. 2018Learning to Estimate 3D Human Pose and Shape from a Single Color Imagecited by 226
  11. 2016Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Posecited by 220
  12. 2021Stochastic Scene-Aware Motion Predictioncited by 203
  13. 2020Long-term Human Motion Prediction with Scene Contextcited by 191
  14. 2020Populating 3D Scenes by Learning Human-Scene Interactioncited by 164
  15. 2020Beyond Static Features for Temporally Consistent 3D Human Pose and Shape from a Videocited by 162
  16. 2021Probabilistic Modeling for Human Mesh Recoverycited by 152
  17. 2022Physical Inertial Poser (PIP): Physics-aware Real-time Human Motion Tracking from Sparse Inertial Sensorscited by 144
  18. 2020Coherent Reconstruction of Multiple Humans from a Single Imagecited by 136
  19. 2020Synthesizing Long-Term 3D Human Motion and Interaction in 3D Scenescited by 135
  20. 2021SPEC: Seeing People in the Wild with an Estimated Cameracited by 134
  21. 2017VNect: Real-time 3D Human Pose Estimation with a Single RGB Cameracited by 129
  22. 2017Towards 3D Human Pose Estimation in the Wild: a Weakly-supervised Approachcited by 114
  23. 20163D Human Pose Estimation = 2D Pose Estimation + Matchingcited by 112
  24. 2021SimPoE: Simulated Character Control for 3D Human Pose Estimationcited by 111
  25. 2021Learning Motion Priors for 4D Human Body Capture in 3D Scenescited by 98
  26. 2020Contact and Human Dynamics from Monocular Videocited by 94
  27. 2021On Self-Contact and Human Posecited by 88
  28. 2019Human Mesh Recovery from Monocular Images via a Skeleton-disentangled Representationcited by 87
  29. 20183D Human Pose Estimation in the Wild by Adversarial Learningcited by 86
  30. 2019Monocular 3D Human Pose Estimation by Generation and Ordinal Rankingcited by 76
  31. 20163D Human Pose Estimation from a Single Image via Distance Matrix Regressioncited by 73
  32. 2017Lifting from the Deep: Convolutional 3D Pose Estimation from a Single Imagecited by 73
  33. 2021Neural Monocular 3D Human Motion Capture with Physical Awarenesscited by 70
  34. 2019XNect: Real-time Multi-Person 3D Motion Capture with a Single RGB Cameracited by 68
  35. 2020PhysCap: Physically Plausible Monocular 3D Motion Capture in Real Timecited by 50
  36. 2019Estimating 3D Motion and Forces of Person-Object Interactions from Monocular Videocited by 48
  37. 2019In the Wild Human Pose Estimation Using Explicit 2D Features and Intermediate 3D Representationscited by 48
  38. 2016Structured Prediction of 3D Human Pose with Deep Neural Networkscited by 43

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 10; 2024: 8; 2025: 3; 2026: 6.

29 papers.

Sort:

By area:

  1. 2022InterCap: Joint Markerless 3D Tracking of Humans and Objects in Interactioncited by 90
  2. 2023Visibility Aware Human-Object Interaction Tracking from Single RGB Cameracited by 58
  3. 2023DECO: Dense Estimation of 3D Human-Scene Contact In The Wildcited by 54
  4. 2024ParaHome: Parameterizing Everyday Home Activities Towards 3D Generative Modeling of Human-Object Interactionscited by 54
  5. 2023LEMON: Learning 3D Human-Object Interaction Relation from 2D Imagescited by 49
  6. 2023Detecting Human-Object Contact in Imagescited by 41
  7. 2023Template Free Reconstruction of Human-object Interaction with Procedural Interaction Generationcited by 37
  8. 2023Probabilistic Human Mesh Recovery in 3D Scenes from Egocentric Viewscited by 33
  9. 2022Scene-aware Egocentric 3D Human Pose Estimationcited by 30
  10. 2024Joint Reconstruction of 3D Human and Object via Contact-Based Refinement Transformercited by 29
  11. 2024EgoChoir: Capturing 3D Human-Object Interaction Regions from Egocentric Viewscited by 26
  12. 2025HOIGen-1M: A Large-scale Dataset for Human-Object Interaction Video Generationcited by 26
17 more
  1. 2023ROAM: Robust and Object-Aware Motion Generation Using Neural Pose Descriptorscited by 23
  2. 2025PICO: Reconstructing 3D People In Contact with Objectscited by 22
  3. 2023Decaf: Monocular Deformation Capture for Face and Hand Interactionscited by 17
  4. 2023Learning Human Mesh Recovery in 3D Scenescited by 13
  5. 2024DICE: End-to-end Deformation Capture of Hand-Face Interactions from a Single Imagecited by 9
  6. 2025End-to-End HOI Reconstruction Transformer with Graph-based Encodingcited by 8
  7. 2023Ins-HOI: Instance Aware Human-Object Interactions Recoverycited by 7
  8. 2024Precision-Enhanced Human-Object Contact Detection via Depth-Aware Perspective Interaction and Object Texture Restorationcited by 5
  9. 20243D Reconstruction of Interacting Multi-Person in Clothing from a Single Imagecited by 4
  10. 2024Purposer: Putting Human Motion Generation in Contextcited by 3
  11. 2024Visual-Geometric Collaborative Guidance for Affordance Learningcited by 2
  12. 2026SceMoS: Scene-Aware 3D Human Motion Synthesis by Planning with Geometry-Grounded Tokenscited by 2
  13. 2026TeHOR: Text-Guided 3D Human and Object Reconstruction with Texturescited by 1
  14. 2026GraphiContact: Pose-aware Human-Scene Robust Contact Perception for Interactive Systemscited by 1
  15. 2026Pi-HOC: Pairwise 3D Human-Object Contact Estimationcited by 0
  16. 2026LEXIS: LatEnt ProXimal Interaction Signatures for 3D HOI from an Imagecited by 0
  17. 2026Reconstructing Humans and Objects in Interaction using Large Reconstruction Modelscited by 0