mlsanityData to 7 Oct 2026

Learning Motion Priors for 4D Human Body Capture in 3D Scenes

arXiv 2108.10399 ↗First posted 23 Aug 2021cs.CV cs.AIcited by 98

Not on the map. Why?

Abstract (arXiv)

Recovering high-quality 3D human motion in complex scenes from monocular videos is important for many applications, ranging from AR/VR to robotics. However, capturing realistic human-scene interactions, while dealing with occlusions and partial views, is challenging; current approaches are still far from achieving compelling results. We address this problem by proposing LEMO: LEarning human MOtion priors for 4D human body capture. By leveraging the large-scale motion capture dataset AMASS, we introduce a novel motion smoothness prior, which strongly reduces the jitters exhibited by poses recovered over a sequence. Furthermore, to handle contacts and occlusions occurring frequently in body-scene interactions, we design a contact friction term and a contact-aware motion infiller obtained via per-instance self-supervised training. To prove the effectiveness of the proposed motion priors, we combine them into a novel pipeline for 4D human body capture in 3D scenes. With our pipeline, we demonstrate high-quality 4D human body capture, reconstructing smooth motions and physically plausible body-scene interactions. The code and data are available at https://sanweiliti.github.io/LEMO/LEMO.html.

Key facts. Cited by 98 papers in the map (data through 7 Oct 2026). Builds on 33 of its 73 references that are in the map. First posted on arXiv 23 Aug 2021.

Builds on

33 of 73 references.

References · 33

  1. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,600
  2. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,208
  3. 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,161
  4. 2017End-to-end Recovery of Human Shape and Posecited by 883
  5. 2019Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loopcited by 625
  6. 2019VIBE: Video Inference for Human Body Pose and Shape Estimationcited by 615
  7. 20183D human pose estimation in video with temporal convolutions and semi-supervised trainingcited by 401
  8. 2018Learning 3D Human Dynamics from Videocited by 315
  9. 2019Resolving 3D Human Pose Ambiguities with 3D Scene Constraintscited by 294
  10. 2018Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodiescited by 239
  11. 2017Single-Shot Multi-Person 3D Pose Estimation From Monocular RGBcited by 195
  12. 2018Neural Body Fitting: Unifying Deep Learning and Model-Based Human Pose and Shape Estimationcited by 185
21 more
  1. 2020Beyond Static Features for Temporally Consistent 3D Human Pose and Shape from a Videocited by 162
  2. 2019Generating 3D People in Scenes without Peoplecited by 127
  3. 2020We are More than Our Joints: Predicting how 3D Bodies Movecited by 115
  4. 2018Human Motion Prediction via Spatio-Temporal Inpaintingcited by 114
  5. 2019Exploiting temporal context for 3D human pose estimation in the wildcited by 110
  6. 2019DenseRaC: Joint 3D Pose and Shape Estimation by Dense Render-and-Comparecited by 100
  7. 20203D Human Motion Estimation via Motion Compression and Refinementcited by 98
  8. 2017Self-supervised Learning of Motion Capturecited by 97
  9. 2020Weakly Supervised 3D Human Pose and Shape Reconstruction with Normalizing Flowscited by 94
  10. 2020Contact and Human Dynamics from Monocular Videocited by 94
  11. 2017Towards Accurate Markerless Human Shape and Pose Estimation over Timecited by 92
  12. 2017Exploiting temporal information for 3D pose estimationcited by 89
  13. 2019Human Mesh Recovery from Monocular Images via a Skeleton-disentangled Representationcited by 87
  14. 2020Motion Guided 3D Pose Estimation from Videoscited by 85
  15. 2020Convolutional Autoencoders for Human Motion Infillingcited by 73
  16. 2015Sparseness Meets Deepness: 3D Human Pose Estimation from Monocular Videocited by 54
  17. 20204D Association Graph for Realtime Multi-person Motion Capture Using Multiple Video Camerascited by 49
  18. 2019Estimating 3D Motion and Forces of Person-Object Interactions from Monocular Videocited by 48
  19. 20204D Human Body Capture from Egocentric Video via 3D Scene Groundingcited by 43
  20. 2020Motion Capture from Internet Videoscited by 33
  21. 2019Spatio-temporal Manifold Learning for Human Motions via Long-horizon Modelingcited by 32

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 22; 2023: 26; 2024: 25; 2025: 12; 2026: 7.

98 papers.

Sort:

By area:

  1. 2022BEHAVE: Dataset and Method for Tracking Human Object Interactionscited by 246
  2. 2023Diffusion-based Generation, Optimization, and Planning in 3D Scenescited by 237
  3. 2023Guided Motion Diffusion for Controllable Human Motion Synthesiscited by 221
  4. 2022Capturing and Inferring Dense Full-Body Human-Scene Contactcited by 176
  5. 2022PyMAF-X: Towards Well-aligned Full-body Model Regression from Monocular Imagescited by 152
  6. 2021GLAMR: Global Occlusion-Aware Human Mesh Recovery with Dynamic Camerascited by 151
  7. 2023Decoupling Human and Camera Motion from Videos in the Wildcited by 140
  8. 2021EgoBody: Human Body Shape and Motion of Interacting People from Head-Mounted Devicescited by 136
  9. 2023Synthesizing Diverse Human Motions in 3D Indoor Scenescited by 109
  10. 2022Recovering 3D Human Mesh from Monocular Images: A Surveycited by 103
  11. 20233D Human Pose Estimation via Intuitive Physicscited by 100
  12. 2023Optimizing Diffusion Noise Can Serve As Universal Motion Priorscited by 97
38 more
  1. 2023Human Motion Generation: A Surveycited by 95
  2. 2022InterCap: Joint Markerless 3D Tracking of Humans and Objects in Interactioncited by 90
  3. 2021SAGA: Stochastic Whole-Body Grasping with Contactcited by 89
  4. 2022FineDance: A Fine-grained Choreography Dataset for 3D Full Body Dance Generationcited by 87
  5. 2022Pose-NDF: Modeling Human Pose Manifolds with Neural Distance Fieldscited by 85
  6. 2022Compositional Human-Scene Interaction Synthesis with Semantic Controlcited by 82
  7. 2022GIMO: Gaze-Informed Human Motion Prediction in Contextcited by 81
  8. 2022Full-Body Articulated Human-Object Interactioncited by 76
  9. 2022TOCH: Spatio-Temporal Object-to-Hand Correspondence for Motion Refinementcited by 69
  10. 2022NeMF: Neural Motion Fields for Kinematic Animationcited by 66
  11. 2021SmoothNet: A Plug-and-Play Network for Refining Human Poses in Videoscited by 64
  12. 2021The Wanderings of Odysseus in 3D Scenescited by 59
  13. 2022COAP: Compositional Articulated Occupancy of Peoplecited by 59
  14. 2024Lodge: A Coarse to Fine Diffusion Network for Long Dance Generation Guided by the Characteristic Dance Primitivescited by 59
  15. 2025Visual Imitation Enables Contextual Humanoid Controlcited by 57
  16. 2022Human-Aware Object Placement for Visual Environment Reconstructioncited by 56
  17. 2023DECO: Dense Estimation of 3D Human-Scene Contact In The Wildcited by 54
  18. 2024RoHM: Robust Human Motion Reconstruction via Diffusioncited by 46
  19. 2023I'M HOI: Inertia-aware Monocular Capture of 3D Human-Object Interactionscited by 42
  20. 2023Detecting Human-Object Contact in Imagescited by 41
  21. 2023Task-Oriented Human-Object Interactions Generation with Implicit Neural Representationscited by 41
  22. 2023Object pop-up: Can we infer 3D objects and their poses from human interactions alone?cited by 36
  23. 2023Probabilistic Human Mesh Recovery in 3D Scenes from Egocentric Viewscited by 33
  24. 2023PACE: Human and Camera Motion Estimation from in-the-wild Videoscited by 33
  25. 2022Neural MoCon: Neural Motion Control for Physically Plausible Human Motion Capturecited by 31
  26. 2022The One Where They Reconstructed 3D Humans and Environments in TV Showscited by 30
  27. 2022HULC: 3D Human Motion Capture with Pose Manifold Sampling and Dense Contact Guidancecited by 29
  28. 2025Joint Optimization for 4D Human-Scene Reconstruction in the Wildcited by 29
  29. 2022Embodied Scene-aware Human Pose Estimationcited by 28
  30. 2022Interaction Replica: Tracking Human-Object Interaction and Scene Changes From Human Motioncited by 27
  31. 2023CIMI4D: A Large Multimodal Climbing Motion Dataset under Human-scene Interactionscited by 27
  32. 2024NRDF: Neural Riemannian Distance Fields for Learning Articulated Pose Priorscited by 27
  33. 2024PhysPT: Physics-aware Pretrained Transformer for Estimating Human Dynamics from Monocular Videoscited by 27
  34. 2025A Survey on Human Interaction Motion Generationcited by 25
  35. 2024F-HOI: Toward Fine-grained Semantic-Aligned 3D Human-Object Interactionscited by 24
  36. 2023HMP: Hand Motion Priors for Pose and Shape Estimation from Videocited by 23
  37. 2024Lodge++: High-quality and Long Dance Generation with Vivid Choreography Patternscited by 21
  38. 2023Scene Synthesis from Human Motioncited by 20

50 most cited of 98 All 98 (JSON)