# HOI-Dyn: Learning Interaction Dynamics for Human-Object Motion Diffusion

arXiv: https://arxiv.org/abs/2507.01737 · HTML: https://mlsanity.com/paper/2507.01737/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 47 references that are in the map.
- First posted on arXiv 2 Jul 2025.

## Builds on (most cited first, 30 of 33)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [Human Motion Diffusion Model](https://mlsanity.com/paper/2209.14916/) (2022)
- [The Replica Dataset: A Digital Replica of Indoor Spaces](https://mlsanity.com/paper/1906.05797/) (2019)
- [MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model](https://mlsanity.com/paper/2208.15001/) (2022)
- [3D-FUTURE: 3D Furniture shape with TextURE](https://mlsanity.com/paper/2009.09633/) (2020)
- [Diffusion-based Generation, Optimization, and Planning in 3D Scenes](https://mlsanity.com/paper/2301.06015/) (2023)
- [Object Motion Guided Human Motion Synthesis](https://mlsanity.com/paper/2309.16237/) (2023)
- [InterDiff: Generating 3D Human-Object Interactions with Physics-Informed Diffusion](https://mlsanity.com/paper/2308.16905/) (2023)
- [HUMANISE: Language-conditioned Human Motion Generation in 3D Scenes](https://mlsanity.com/paper/2210.09729/) (2022)
- [CG-HOI: Contact-Guided 3D Human-Object Interaction Generation](https://mlsanity.com/paper/2311.16097/) (2023)
- [Synthesizing Diverse Human Motions in 3D Indoor Scenes](https://mlsanity.com/paper/2305.12411/) (2023)
- [Controllable Human-Object Interaction Synthesis](https://mlsanity.com/paper/2312.03913/) (2023)
- [Efficient Learning on Point Clouds with Basis Point Sets](https://mlsanity.com/paper/1908.09186/) (2019)
- [Human Motion Generation: A Survey](https://mlsanity.com/paper/2307.10894/) (2023)
- [HOI-Diff: Text-Driven Synthesis of 3D Human-Object Interactions using Diffusion Models](https://mlsanity.com/paper/2312.06553/) (2023)
- [Text2HOI: Text-guided 3D Motion Generation for Hand-Object Interaction](https://mlsanity.com/paper/2404.00562/) (2024)
- [Move as You Say, Interact as You Can: Language-guided Human Motion Generation with Scene Affordance](https://mlsanity.com/paper/2403.18036/) (2024)
- [PhysHOI: Physics-Based Imitation of Dynamic Human-Object Interaction](https://mlsanity.com/paper/2312.04393/) (2023)
- [InterMimic: Towards Universal Whole-Body Control for Physics-Based Human-Object Interactions](https://mlsanity.com/paper/2502.20390/) (2025)
- [Human-Object Interaction from Human-Level Instructions](https://mlsanity.com/paper/2406.17840/) (2024)
- [Generating Human Motion in 3D Scenes from Text Descriptions](https://mlsanity.com/paper/2405.07784/) (2024)
- [THOR: Text to Human-Object Interaction Diffusion via Relation Intervention](https://mlsanity.com/paper/2403.11208/) (2024)
- [Hierarchical Generation of Human-Object Interactions with Diffusion Probabilistic Models](https://mlsanity.com/paper/2310.02242/) (2023)
- [HIMO: A New Benchmark for Full-Body Human Interacting with Multiple Objects](https://mlsanity.com/paper/2407.12371/) (2024)
- [GeneOH Diffusion: Towards Generalizable Hand-Object Interaction Denoising via Denoising Diffusion](https://mlsanity.com/paper/2402.14810/) (2024)
- [ChainHOI: Joint-based Kinematic Chain Modeling for Human-Object Interaction Generation](https://mlsanity.com/paper/2503.13130/) (2025)
- [Closely Interactive Human Reconstruction with Proxemics and Physics-Guided Adaption](https://mlsanity.com/paper/2404.11291/) (2024)

## Built on by (most cited first, 10 of 10)

- [InterPrior: Scaling Generative Control for Physics-Based Human-Object Interactions](https://mlsanity.com/paper/2602.06035/) (2026)
- [Advances in 4D Representation: Geometry, Motion, and Interaction](https://mlsanity.com/paper/2510.19255/) (2025)
- [ViHOI: Human-Object Interaction Synthesis with Visual Priors](https://mlsanity.com/paper/2603.24383/) (2026)
- [Not All Frames Are Equal: Complexity-Aware Masked Motion Generation via Motion Spectral Descriptors](https://mlsanity.com/paper/2603.29655/) (2026)
- [HOIMask: Towards Generative Masked Modeling for Human Object Interaction Generation](https://mlsanity.com/paper/2608.15141/) (2026)
- [Stability-Driven Motion Generation for Object-Guided Human-Human Co-Manipulation](https://mlsanity.com/paper/2604.20336/) (2026)
- [MaMi-HOI: Harmonizing Global Kinematics and Local Geometry for Human-Object Interaction Generation](https://mlsanity.com/paper/2605.05756/) (2026)
- [Dynamic Full-body Motion Agent with Object Interaction via Blending Pre-trained Modular Controllers](https://mlsanity.com/paper/2605.11369/) (2026)
- [LYRIC: Language-Driven Physics-Based Character Control for Contact-Rich Whole-Body Object Interaction](https://mlsanity.com/paper/2609.19688/) (2026)
- [PAMI: Part Anchored Motion for Text to Human-Object Interaction Generation](https://mlsanity.com/paper/2609.38466/) (2026)

## Abstract (arXiv)

Generating realistic 3D human-object interactions (HOIs) remains a challenging task due to the difficulty of modeling detailed interaction dynamics. Existing methods treat human and object motions independently, resulting in physically implausible and causally inconsistent behaviors. In this work, we present HOI-Dyn, a novel framework that formulates HOI generation as a driver-responder system, where human actions drive object responses. At the core of our method is a lightweight transformer-based interaction dynamics model that explicitly predicts how objects should react to human motion. To further enforce consistency, we introduce a residual-based dynamics loss that mitigates the impact of dynamics prediction errors and prevents misleading optimization signals. The dynamics model is used only during training, preserving inference efficiency. Through extensive qualitative and quantitative experiments, we demonstrate that our approach not only enhances the quality of HOI generation but also establishes a feasible metric for evaluating the quality of generated interactions.