mlsanityData to 7 Oct 2026

D3D-HOI: Dynamic 3D Human-Object Interactions from Videos

arXiv 2108.08420 ↗First posted 19 Aug 2021cs.CVcited by 39

Not on the map. Why?

Abstract (arXiv)

We introduce D3D-HOI: a dataset of monocular videos with ground truth annotations of 3D object pose, shape and part motion during human-object interactions. Our dataset consists of several common articulated objects captured from diverse real-world scenes and camera viewpoints. Each manipulated object (e.g., microwave oven) is represented with a matching 3D parametric model. This data allows us to evaluate the reconstruction quality of articulated objects and establish a benchmark for this challenging task. In particular, we leverage the estimated 3D human pose for more accurate inference of the object spatial layout and dynamics. We evaluate this approach on our dataset, demonstrating that human-object relations can significantly reduce the ambiguity of articulated object reconstructions from challenging real-world videos. Code and dataset are available at https://github.com/facebookresearch/d3d-hoi.

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). Builds on 27 of its 37 references that are in the map. First posted on arXiv 19 Aug 2021.

Builds on

27 of 37 references.

References · 27

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,600
  3. 2017End-to-end Recovery of Human Shape and Posecited by 883
  4. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 781
  5. 2016Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Imagecited by 692
  6. 2019Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loopcited by 625
  7. 2019VIBE: Video Inference for Human Body Pose and Shape Estimationcited by 615
  8. 2020SAPIEN: A SimulAted Part-based Interactive ENvironmentcited by 584
  9. 2019PointRend: Image Segmentation as Renderingcited by 439
  10. 2019Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoningcited by 420
  11. 2019Resolving 3D Human Pose Ambiguities with 3D Scene Constraintscited by 294
  12. 2018Learning to Estimate 3D Human Pose and Shape from a Single Color Imagecited by 226
15 more
  1. 2018Neural Body Fitting: Unifying Deep Learning and Model-Based Human Pose and Shape Estimationcited by 185
  2. 2020Grasping Field: Learning Implicit Representations for Human Graspscited by 178
  3. 2020Total3DUnderstanding: Joint Layout, Object Pose and Mesh Reconstruction for Indoor Scenes from a Single Imagecited by 170
  4. 2020Populating 3D Scenes by Learning Human-Scene Interactioncited by 164
  5. 2019Category-Level Articulated Object Pose Estimationcited by 158
  6. 2020Perceiving 3D Human-Object Spatial Arrangements from a Single Image in the Wildcited by 152
  7. 2019Generating 3D People in Scenes without Peoplecited by 127
  8. 20203D Human Motion Estimation via Motion Compression and Refinementcited by 98
  9. 2019Holistic++ Scene Understanding: Single-view 3D Holistic Scene Parsing and Human Pose Estimation with Human-Object Interaction and Physical Commonsensecited by 71
  10. 2020ScrewNet: Category-Independent Articulation Model Estimation From Depth Images Using Screw Theorycited by 70
  11. 2020Holistic 3D Human and Scene Mesh Estimation from Single View Imagescited by 54
  12. 2017Factoring Shape, Pose, and Layout from the 2D Image of a 3D Scenecited by 50
  13. 2021MultiBodySync: Multi-Body Segmentation and Motion Estimation via 3D Scan Synchronizationcited by 46
  14. 2020Vid2CAD: CAD Model Alignment using Multi-View Constraints from Videoscited by 33
  15. 20193D-RelNet: Joint Object and Relational Network for 3D Predictioncited by 28

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 7; 2023: 10; 2024: 11; 2025: 6; 2026: 5.

39 papers.

Sort:

By area:

  1. 2023InterDiff: Generating 3D Human-Object Interactions with Physics-Informed Diffusioncited by 181
  2. 2022EPIC-KITCHENS VISOR Benchmark: VIdeo Segmentations and Object Relationscited by 139
  3. 2022IMos: Intent-Driven Full-Body Motion Synthesis for Human-Object Interactionscited by 121
  4. 2022Full-Body Articulated Human-Object Interactioncited by 76
  5. 2024InterDreamer: Zero-Shot Text to 3D Dynamic Human-Object Interactioncited by 63
  6. 2023Visibility Aware Human-Object Interaction Tracking from Single RGB Cameracited by 58
  7. 2023ArtiGrasp: Physically Plausible Synthesis of Bi-Manual Dexterous Grasping and Articulationcited by 53
  8. 2024HOI-M3:Capture Multiple Humans and Objects Interaction within Contextual Environmentcited by 52
  9. 2023Locomotion-Action-Manipulation: Synthesizing Human-Scene Interactions in Complex 3D Environmentscited by 51
  10. 2023LEMON: Learning 3D Human-Object Interaction Relation from 2D Imagescited by 49
  11. 2023Template Free Reconstruction of Human-object Interaction with Procedural Interaction Generationcited by 37
  12. 2022Understanding 3D Object Articulation in Internet Videoscited by 36
27 more
  1. 2024HIMO: A New Benchmark for Full-Body Human Interacting with Multiple Objectscited by 34
  2. 2025InterAct: Advancing Large-Scale Versatile 3D Human-Object Interaction Generationcited by 31
  3. 2022The One Where They Reconstructed 3D Humans and Environments in TV Showscited by 30
  4. 2024Joint Reconstruction of 3D Human and Object via Contact-Based Refinement Transformercited by 29
  5. 2022Interaction Replica: Tracking Human-Object Interaction and Scene Changes From Human Motioncited by 27
  6. 2024F-HOI: Toward Fine-grained Semantic-Aligned 3D Human-Object Interactionscited by 24
  7. 2024InterTrack: Tracking Human Object Interaction without Object Templatescited by 22
  8. 2024Survey on Modeling of Human-made Articulated Objectscited by 14
  9. 2025Reconstructing 4D Spatial Intelligence: A Surveycited by 13
  10. 2023Compositional 3D Human-Object Neural Animationcited by 9
  11. 2024DreamHOI: Subject-Driven Generation of 3D Human-Object Interactions with Diffusion Priorscited by 9
  12. 2022Articulated 3D Human-Object Interactions from RGB Videos: An Empirical Analysis of Approaches and Challengescited by 8
  13. 2023Ins-HOI: Instance Aware Human-Object Interactions Recoverycited by 7
  14. 2023Primitive-based 3D Human-Object Interaction Modelling and Programmingcited by 7
  15. 2024FORCE: Physics-aware Human-object Interactioncited by 7
  16. 2025Perceiving and Acting in First-Person: A Dataset and Benchmark for Egocentric Human-Object-Human Interactionscited by 7
  17. 2026OmniShow: Unifying Multimodal Conditions for Human-Object Interaction Video Generationcited by 7
  18. 2023Doduo: Learning Dense Visual Correspondence from Unsupervised Semantic-Aware Flowcited by 6
  19. 20253D Human Interaction Generation: A Surveycited by 6
  20. 2025GRACE: Estimating Geometry-level 3D Human-Scene Contact from 2D Imagescited by 3
  21. 2024COLLAGE: Collaborative Human-Agent Interaction Generation using Hierarchical Latent Diffusion and Language Modelscited by 2
  22. 2026TeHOR: Text-Guided 3D Human and Object Reconstruction with Texturescited by 1
  23. 2026PAWS: Perception of Articulation in the Wild at Scale from Egocentric Videoscited by 1
  24. 2024A New People-Object Interaction Dataset and NVS Benchmarkscited by 0
  25. 2025Realistic Clothed Human and Object Joint Reconstruction from a Single Imagecited by 0
  26. 2026ArtHOI: Articulated Human-Object Interaction Synthesis by 4D Reconstruction from Video Priorscited by 0
  27. 2026SynAgent: Generalizable Cooperative Humanoid Manipulation via Solo-to-Cooperative Agent Synergycited by 0