mlsanityData to 14 Oct 2026

InterDiff: Generating 3D Human-Object Interactions with Physics-Informed Diffusion

arXiv 2308.16905 ↗First posted 31 Aug 2023cs.CV cs.AI cs.GRcited by 182

Not on the map. Why?

Cited by 38 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

This paper addresses a novel task of anticipating 3D human-object interactions (HOIs). Most existing research on HOI synthesis lacks comprehensive whole-body interactions with dynamic objects, e.g., often limited to manipulating small or static objects. Our task is significantly more challenging, as it requires modeling dynamic objects with various shapes, capturing whole-body motion, and ensuring physically valid interactions. To this end, we propose InterDiff, a framework comprising two key steps: (i) interaction diffusion, where we leverage a diffusion model to encode the distribution of future human-object interactions; (ii) interaction correction, where we introduce a physics-informed predictor to correct denoised HOIs in a diffusion step. Our key insight is to inject prior knowledge that the interactions under reference with respect to contact points follow a simple pattern and are easily predictable. Experiments on multiple human-object interaction datasets demonstrate the effectiveness of our method for this task, capable of producing realistic, vivid, and remarkably long-term 3D HOI predictions.

Key facts. Cited by 182 papers in the map (data through 14 Oct 2026). Builds on 100 of its 136 references that are in the map. First posted on arXiv 31 Aug 2023.

Builds on

100 of 136 references.

References · 100

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2020Denoising Diffusion Implicit Modelscited by 9,781
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  6. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  7. 2013Auto-Encoding Variational Bayescited by 4,053
  8. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,689
  9. 2015Variational Inference with Normalizing Flowscited by 2,170
  10. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,214
  11. 2022Human Motion Diffusion Modelcited by 1,080
  12. 2022MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Modelcited by 780
38 more
  1. 2022Compositional Visual Generation with Composable Diffusion Modelscited by 624
  2. 2023T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representationscited by 588
  3. 2022Executing your Commands via Motion Diffusion in Latent Spacecited by 575
  4. 2021Action-Conditioned 3D Human Motion Synthesis with Transformer VAEcited by 501
  5. 2022TEMOS: Generating diverse human motions from textual descriptionscited by 492
  6. 2022MotionCLIP: Exposing Human Motion Generation to CLIP Spacecited by 408
  7. 2020GRAB: A Dataset of Whole-Body Human Grasping of Objectscited by 405
  8. 2022PhysDiff: Physics-Guided Human Motion Diffusion Modelcited by 368
  9. 2023Human Motion Diffusion as a Generative Priorcited by 357
  10. 2022ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulationcited by 312
  11. 2023ReMoDiffuse: Retrieval-Augmented Motion Diffusion Modelcited by 287
  12. 2022BEHAVE: Dataset and Method for Tracking Human Object Interactionscited by 249
  13. 2019Learning Trajectory Dependencies for Human Motion Predictioncited by 246
  14. 2023Diffusion-based Generation, Optimization, and Planning in 3D Scenescited by 239
  15. 2016Deep Unsupervised Clustering with Gaussian Mixture Variational Autoencoderscited by 220
  16. 2022MoFusion: A Framework for Denoising-Diffusion-based Motion Synthesiscited by 218
  17. 2021Stochastic Scene-Aware Motion Predictioncited by 203
  18. 2017Detecting and Recognizing Human-Object Interactionscited by 199
  19. 2020Long-term Human Motion Prediction with Scene Contextcited by 192
  20. 2020DLow: Diversifying Latent Flows for Diverse Human Motion Predictioncited by 185
  21. 2020Grasping Field: Learning Implicit Representations for Human Graspscited by 179
  22. 2020Populating 3D Scenes by Learning Human-Scene Interactioncited by 164
  23. 2021Hand-Object Contact Consistency Reasoning for Human Grasps Generationcited by 160
  24. 2022HUMANISE: Language-conditioned Human Motion Generation in 3D Scenescited by 158
  25. 2020Perceiving 3D Human-Object Spatial Arrangements from a Single Image in the Wildcited by 152
  26. 2021GOAL: Generating 4D Whole-Body Motion for Hand-Object Graspingcited by 144
  27. 2021ContactOpt: Optimizing Contact to Improve Graspscited by 139
  28. 2020Synthesizing Long-Term 3D Human Motion and Interaction in 3D Scenescited by 135
  29. 2022COUCH: Towards Controllable Human-Chair Interactionscited by 135
  30. 2022IMos: Intent-Driven Full-Body Motion Synthesis for Human-Object Interactionscited by 124
  31. 2020We are More than Our Joints: Predicting how 3D Bodies Movecited by 115
  32. 2022Joint Hand Motion and Interaction Hotspots Prediction from Egocentric Videoscited by 111
  33. 2023Affordance Diffusion: Synthesizing Hand-Object Interactionscited by 110
  34. 2023Synthesizing Diverse Human Motions in 3D Indoor Scenescited by 109
  35. 2023Synthesizing Physical Character-Scene Interactionscited by 104
  36. 2022CHORE: Contact, Human and Object REconstruction from a single RGB imagecited by 95
  37. 2022InterCap: Joint Markerless 3D Tracking of Humans and Objects in Interactioncited by 93
  38. 2018Flexible Neural Representation for Physics Predictioncited by 91

50 most cited of 100 All 100 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 14; 2024: 56; 2025: 70; 2026: 42.

182 papers.

Sort:

By area:

  1. 2023OmniControl: Control Any Joint at Any Time for Human Motion Generationcited by 238
  2. 2024VideoPhy: Evaluating Physical Commonsense for Video Generationcited by 228
  3. 2024MotionLCM: Real-time Controllable Motion Generation via Latent Consistency Modelcited by 168
  4. 2023CG-HOI: Contact-Guided 3D Human-Object Interaction Generationcited by 138
  5. 2024Scaling Up Dynamic Human-Scene Interaction Modelingcited by 137
  6. 2024MaskedMimic: Unified Physics-Based Character Control Through Masked Motion Inpaintingcited by 116
  7. 2023Controllable Human-Object Interaction Synthesiscited by 109
  8. 2023HumanTOMATO: Text-aligned Whole-body Motion Generationcited by 104
  9. 2023HOI-Diff: Text-Driven Synthesis of 3D Human-Object Interactions using Diffusion Modelscited by 93
  10. 2024TACO: Benchmarking Generalizable Bimanual Tool-ACtion-Object Understandingcited by 88
  11. 2024Rethinking Diffusion for Text-Driven Human Motion Generation: Redundant Representations, Evaluation, and Masked Autoregressioncited by 85
  12. 2024DartControl: A Diffusion-Based Autoregressive Motion Model for Real-Time Text-Driven Motion Controlcited by 84
38 more
  1. 2024Seamless Human Motion Composition with Blended Positional Encodingscited by 80
  2. 2023PhysHOI: Physics-Based Imitation of Dynamic Human-Object Interactioncited by 76
  3. 2025InterMimic: Towards Universal Whole-Body Control for Physics-Based Human-Object Interactionscited by 72
  4. 2024InterDreamer: Zero-Shot Text to 3D Dynamic Human-Object Interactioncited by 64
  5. 2024Human-Object Interaction from Human-Level Instructionscited by 64
  6. 2023ReMoS: 3D Motion-Conditioned Reaction Synthesis for Two-Person Interactionscited by 62
  7. 2024DiffH2O: Diffusion-Based Synthesis of Hand-Object Interactions from Textual Descriptionscited by 62
  8. 2024ParaHome: Parameterizing Everyday Home Activities Towards 3D Generative Modeling of Human-Object Interactionscited by 57
  9. 2025TokenHSI: Unified Synthesis of Physical Human-Scene Interactions through Task Tokenizationcited by 51
  10. 2023LEMON: Learning 3D Human-Object Interaction Relation from 2D Imagescited by 49
  11. 2024Generating Human Motion in 3D Scenes from Text Descriptionscited by 49
  12. 2024THOR: Text to Human-Object Interaction Diffusion via Relation Interventioncited by 46
  13. 2023I'M HOI: Inertia-aware Monocular Capture of 3D Human-Object Interactionscited by 44
  14. 2024CORE4D: A 4D Human-Object-Human Interaction Dataset for Collaborative Object REarrangementcited by 44
  15. 2024Self-Correcting Self-Consuming Loops for Generative Model Trainingcited by 37
  16. 2024SMooDi: Stylized Motion Diffusion Modelcited by 37
  17. 2024HIMO: A New Benchmark for Full-Body Human Interacting with Multiple Objectscited by 36
  18. 2024LaserHuman: Language-guided Scene-aware Human Motion Generation in Free Environmentcited by 33
  19. 2024GigaHands: A Massive Annotated Dataset of Bimanual Hand Activitiescited by 33
  20. 2024Beyond the Contact: Discovering Comprehensive Affordance for 3D Objects from Pre-trained 2D Diffusion Modelscited by 32
  21. 2024AnySkill: Learning Open-Vocabulary Physical Skill for Interactive Agentscited by 32
  22. 2025InterAct: Advancing Large-Scale Versatile 3D Human-Object Interaction Generationcited by 32
  23. 2024Harmony4D: A Video Dataset for In-The-Wild Close Human Interactionscited by 31
  24. 2025HUMOTO: A 4D Dataset of Mocap Human Object Interactionscited by 31
  25. 2024SkillMimic: Learning Basketball Interaction Skills from Demonstrationscited by 30
  26. 2024Unimotion: Unifying 3D Human Motion Synthesis and Understandingcited by 30
  27. 2024BimArt: A Unified Approach for the Synthesis of 3D Bimanual Interaction with Articulated Objectscited by 30
  28. 2025Absolute Coordinates Make Motion Generation Easycited by 29
  29. 2023Interactive Humanoid: Online Full-Body Motion Reaction Synthesis with Social Affordance Canonicalization and Forecastingcited by 27
  30. 2025SimMotionEdit: Text-Based Human Motion Editing with Motion Similarity Predictioncited by 27
  31. 2024EgoChoir: Capturing 3D Human-Object Interaction Regions from Egocentric Viewscited by 26
  32. 2024ZeroHSI: Zero-Shot 4D Human-Scene Interaction by Video Generationcited by 25
  33. 2025A Survey on Human Interaction Motion Generationcited by 25
  34. 2025SkillMimic-V2: Learning Robust and Generalizable Interaction Skills from Sparse and Noisy Demonstrationscited by 25
  35. 2024F-HOI: Toward Fine-grained Semantic-Aligned 3D Human-Object Interactionscited by 24
  36. 2025Guiding Human-Object Interactions with Rich Geometry and Relationscited by 24
  37. 2023InterControl: Zero-shot Human Interaction Generation by Controlling Every Jointcited by 23
  38. 2025DreamActor-H1: High-Fidelity Human-Product Demonstration Video Generation via Motion-designed Diffusion Transformerscited by 22

50 most cited of 182 All 182 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025HOIDiNi: Human-Object Interaction through Diffusion Noise Optimization
  2. 2024OOD-HOI: Text-Driven 3D Whole-Body Human-Object Interactions Generation Beyond Training Domains
  3. 2023Hierarchical Generation of Human-Object Interactions with Diffusion Probabilistic Models
  4. 2023Template Free Reconstruction of Human-object Interaction with Procedural Interaction Generation
  5. 2024DreamHOI: Subject-Driven Generation of 3D Human-Object Interactions with Diffusion Priors
  6. 2026OneHOI: Unifying Human-Object Interaction Generation and Editing
  7. 2024AvatarGO: Zero-shot 4D Human-Object Interaction Generation and Animation
  8. 2026PhyGenHOI: Physically-Aware 4D Generation of Dynamic Human-Object Interactions