mlsanityData to 14 Oct 2026

Articulated 3D Human-Object Interactions from RGB Videos: An Empirical Analysis of Approaches and Challenges

arXiv 2209.05612 ↗First posted 12 Sep 2022cs.CVcited by 8

Not on the map. Why?

Abstract (arXiv)

Human-object interactions with articulated objects are common in everyday life. Despite much progress in single-view 3D reconstruction, it is still challenging to infer an articulated 3D object model from an RGB video showing a person manipulating the object. We canonicalize the task of articulated 3D human-object interaction reconstruction from RGB video, and carry out a systematic benchmark of five families of methods for this task: 3D plane estimation, 3D cuboid estimation, CAD model fitting, implicit field fitting, and free-form mesh fitting. Our experiments show that all methods struggle to obtain high accuracy results even when provided ground truth information about the observed objects. We identify key factors which make the task challenging and suggest directions for future work on this challenging 3D computer vision task. Short video summary at https://www.youtube.com/watch?v=5tAlKBojZwc

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 38 of its 52 references that are in the map. First posted on arXiv 12 Sep 2022.

Builds on

38 of 52 references.

References · 38

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2017Mask R-CNNcited by 7,277
  3. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  4. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,612
  5. 2021Per-Pixel Classification is Not All You Need for Semantic Segmentationcited by 1,152
  6. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 979
  7. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 784
  8. 2017PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenescited by 775
  9. 2016Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Imagecited by 695
  10. 2019Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimationcited by 495
  11. 2019Learning joint reconstruction of hands and manipulated objectscited by 378
  12. 2018Learning 3D Human Dynamics from Videocited by 316
26 more
  1. 20163D Menagerie: Modeling the 3D shape and pose of animalscited by 312
  2. 2022BEHAVE: Dataset and Method for Tracking Human Object Interactionscited by 249
  3. 2021BANMo: Building Animatable 3D Neural Models from Many Casual Videoscited by 197
  4. 2020Populating 3D Scenes by Learning Human-Scene Interactioncited by 164
  5. 2019Category-Level Articulated Object Pose Estimationcited by 159
  6. 2022Ditto: Building Digital Twins of Articulated Objects from Interactioncited by 156
  7. 2020Perceiving 3D Human-Object Spatial Arrangements from a Single Image in the Wildcited by 152
  8. 2020Reconstructing Hand-Object Interactions in the Wildcited by 144
  9. 2019Shape2Motion: Joint Analysis of Motion Parts and Attributes from 3D Shapescited by 119
  10. 2021LASR: Learning Articulated Shape Reconstruction from a Monocular Videocited by 116
  11. 2015Capturing Hands in Action using Discriminative Salient Points and Physics Simulationcited by 115
  12. 2021A-SDF: Learning Disentangled Signed Distance Functions for Articulated Shape Representationcited by 114
  13. 2019Three-D Safari: Learning to Estimate Zebra Pose, Shape, and Texture from Images "In the Wild"cited by 106
  14. 2022CHORE: Contact, Human and Object REconstruction from a single RGB imagecited by 95
  15. 2018Creatures great and SMAL: Recovering the shape and motion of animals from videocited by 84
  16. 2020ScrewNet: Category-Independent Articulation Model Estimation From Depth Images Using Screw Theorycited by 70
  17. 2021DOVE: Learning Deformable 3D Objects by Watching Videoscited by 61
  18. 2022CLA-NeRF: Category-Level Articulated Neural Radiance Fieldcited by 56
  19. 2021Watch It Move: Unsupervised Discovery of 3D Joints for Re-Posing of Articulated Objectscited by 50
  20. 2014A Probabilistic Framework for Learning Kinematic Models of Articulated Objectscited by 40
  21. 2021D3D-HOI: Dynamic 3D Human-Object Interactions from Videoscited by 39
  22. 2022Understanding 3D Object Articulation in Internet Videoscited by 36
  23. 2021KAMA: 3D Keypoint Aware Body Mesh Articulationcited by 27
  24. 2020Nothing But Geometric Constraints: A Model-Free Method for Articulated Object Pose Estimationcited by 25
  25. 2021Distributional Depth-Based Estimation of Object Articulation Modelscited by 18
  26. 2020Human-Object Interaction Detection:A Quick Survey and Examination of Methodscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 4; 2024: 1; 2025: 1; 2026: 1.

8 papers.

  1. 2023PARIS: Part-level Reconstruction and Motion Analysis for Articulated Objectscited by 118
  2. 2022Full-Body Articulated Human-Object Interactioncited by 78
  3. 2023Understanding 3D Object Interaction from a Single Imagecited by 33
  4. 2023Instant-NVR: Instant Neural Volumetric Rendering for Human-object Interactions from Monocular RGBD Streamcited by 23
  5. 2023Compositional 3D Human-Object Neural Animationcited by 9
  6. 2025MonoMobility: Zero-Shot 3D Mobility Analysis from Monocular Videoscited by 6
  7. 2024PAPR in Motion: Seamless Point-level 3D Scene Interpolationcited by 5
  8. 2026PAWS: Perception of Articulation in the Wild at Scale from Egocentric Videoscited by 2