# AgentHOI: Multi-Agent Reasoning for Human-Object-Interaction Video Generation via Implicit Representation Alignment

arXiv: https://arxiv.org/abs/2607.22241 · HTML: https://mlsanity.com/paper/2607.22241/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Video generation, one of 14 directions in Diffusion models.
- Builds on 41 of its 73 references that are in the map.
- First posted on arXiv 24 Jul 2026.

## Placed in

- Diffusion models / [Video generation](https://mlsanity.com/diffusion/video-generation/)

## Builds on (most cited first, 30 of 41)

- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models](https://mlsanity.com/paper/2504.10479/) (2025)
- [HunyuanVideo: A Systematic Framework For Large Video Generative Models](https://mlsanity.com/paper/2412.03603/) (2024)
- [FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Space](https://mlsanity.com/paper/2506.15742/) (2025)
- [Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think](https://mlsanity.com/paper/2410.06940/) (2024)
- [VACE: All-in-One Video Creation and Editing](https://mlsanity.com/paper/2503.07598/) (2025)
- [Improving Video Generation with Human Feedback](https://mlsanity.com/paper/2501.13918/) (2025)
- [Z-Image: An Efficient Image Generation Foundation Model with Single-Stream Diffusion Transformer](https://mlsanity.com/paper/2511.22699/) (2025)
- [MimicMotion: High-Quality Human Motion Video Generation with Confidence-aware Pose Guidance](https://mlsanity.com/paper/2406.19680/) (2024)
- [OmniHuman-1: Rethinking the Scaling-Up of One-Stage Conditioned Human Animation Models](https://mlsanity.com/paper/2502.01061/) (2025)
- [UniAnimate: Taming Unified Video Diffusion Models for Consistent Human Image Animation](https://mlsanity.com/paper/2406.01188/) (2024)
- [VideoREPA: Learning Physics for Video Generation through Relational Alignment with Foundation Models](https://mlsanity.com/paper/2505.23656/) (2025)
- [HOI-Diff: Text-Driven Synthesis of 3D Human-Object Interactions using Diffusion Models](https://mlsanity.com/paper/2312.06553/) (2023)
- [StableAnimator: High-Quality Identity-Preserving Human Image Animation](https://mlsanity.com/paper/2411.17697/) (2024)
- [Wan-Animate: Unified Character Animation and Replacement with Holistic Replication](https://mlsanity.com/paper/2509.14055/) (2025)
- [EchoMimicV2: Towards Striking, Simplified, and Semi-Body Human Animation](https://mlsanity.com/paper/2411.10061/) (2024)
- [Sonic: Shifting Focus to Global Audio Perception in Portrait Animation](https://mlsanity.com/paper/2411.16331/) (2024)
- [Allegro: Open the Black Box of Commercial-Level Video Generation Model](https://mlsanity.com/paper/2410.15458/) (2024)
- [HunyuanVideo-Avatar: High-Fidelity Audio-Driven Human Animation for Multiple Characters](https://mlsanity.com/paper/2505.20156/) (2025)
- [Animate Anyone 2: High-Fidelity Character Image Animation with Environment Affordance](https://mlsanity.com/paper/2502.06145/) (2025)
- [VLOGGER: Multimodal Diffusion for Embodied Avatar Synthesis](https://mlsanity.com/paper/2403.08764/) (2024)
- [HuMo: Human-Centric Video Generation via Collaborative Multi-Modal Conditioning](https://mlsanity.com/paper/2509.08519/) (2025)
- [Mora: Enabling Generalist Video Generation via A Multi-Agent Framework](https://mlsanity.com/paper/2403.13248/) (2024)
- [UniAnimate-DiT: Human Image Animation with Large-Scale Video Diffusion Transformer](https://mlsanity.com/paper/2504.11289/) (2025)
- [CyberHost: Taming Audio-driven Avatar Diffusion Model with Region Codebook Attention](https://mlsanity.com/paper/2409.01876/) (2024)

## Built on by (most cited first, 1 of 1)

- [Agentic Visual Generation: From Generative Models to Agentic Control](https://mlsanity.com/paper/2609.06758/) (2026)

## Abstract (arXiv)

Recent advances in video diffusion models have spurred interest in human-object interaction (HOI) video generation, which demands fine-grained control over interaction logic beyond single-subject animation. However, existing HOI methods rely heavily on explicit motion control, limiting scalability and generalization across diverse objects and interactions. In this study, we propose AgentHOI, a text-driven HOI video generation following a thinking-before-generation framework that bridges the gap between high-level textual intent and physical execution through multi-agent reasoning over perception, interaction, and motion planning. Building upon the generated interaction plans, we further strengthen text-driven motion understanding. We introduce an implicit text-motion alignment strategy that distills text-to-motion priors into the video diffusion model, enabling robust HOI synthesis without explicit motion inputs at inference. Experiments show that AgentHOI significantly improves interaction naturalness, object appearance preservation, and adherence to complex textual instructions across challenging object-centric scenarios such as wearing and riding. The code is available at https://github.com/bone-11/agenthoi.