GenHOI: Towards Object-Consistent Hand-Object Interaction with Temporally Balanced and Spatially Selective Object Injection
Not on the map. Why?
Abstract (arXiv)
Hand-Object Interaction (HOI) remains a core challenge in digital human video synthesis, where models must generate physically plausible contact and preserve object identity across frames. Although recent HOI reenactment approaches have achieved progress, they are typically trained and evaluated in-domain and fail to generalize to complex, in-the-wild scenarios. In contrast, all-in-one video editing models exhibit broader robustness but still struggle with HOI-specific issues such as inconsistent object appearance. In this paper, we present GenHOI, a lightweight augmentation to pretrained video generation models that injects reference-object information in a temporally balanced and spatially selective manner. For temporal balancing, we propose Head-Sliding RoPE, which assigns head-specific temporal offsets to reference tokens, distributing their influence evenly across frames and mitigating the temporal decay of 3D RoPE to improve long-range object consistency. For spatial selectivity, we design a two-level spatial attention gate that concentrates object-conditioned attention on HOI regions and adaptively scales its strength, preserving background realism while enhancing interaction fidelity. Extensive qualitative and quantitative evaluations on unseen, in-the-wild scenes demonstrate that GenHOI significantly outperforms state-of-the-art HOI reenactment and all-in-one video editing methods. Project page: https://xuanhuang0.github.io/GenHOI/
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 54 of its 62 references that are in the map. First posted on arXiv 6 Mar 2026.
Builds on
54 of 62 references.
References · 54
- 2017Attention Is All You Needcited by 59,932
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
- 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
- 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
- 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
- 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 960
- 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 958
- 2023Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animationcited by 683
38 more
- 2020A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wildcited by 669
- 2025VACE: All-in-One Video Creation and Editingcited by 467
- 2023FateZero: Fusing Attentions for Zero-shot Text-based Video Editingcited by 462
- 2023TokenFlow: Consistent Diffusion Features for Consistent Video Editingcited by 440
- 2020One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencingcited by 438
- 2024OminiControl: Minimal and Universal Control for Diffusion Transformercited by 334
- 2023Pix2Video: Video Editing using Image Diffusioncited by 318
- 2023Video-P2P: Video Editing with Cross-attention Controlcited by 312
- 2021Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representationcited by 288
- 2025Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Freecited by 266
- 2023Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioningcited by 254
- 2023Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translationcited by 253
- 2024EMO: Emote Portrait Alive -- Generating Expressive Portrait Videos with Audio2Video Diffusion Model under Weak Conditionscited by 233
- 2024LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Controlcited by 202
- 2024Hallo: Hierarchical Audio-Driven Visual Synthesis for Portrait Image Animationcited by 198
- 2024MimicMotion: High-Quality Human Motion Video Generation with Confidence-aware Pose Guidancecited by 196
- 2022Latent Image Animator: Learning to Animate Images via Latent Space Navigationcited by 172
- 2023DreamPose: Fashion Image-to-Video Synthesis via Stable Diffusioncited by 167
- 2017Video Generation From Textcited by 153
- 2023DisCo: Disentangled Control for Realistic Human Dance Generationcited by 141
- 2023CG-HOI: Contact-Guided 3D Human-Object Interaction Generationcited by 137
- 2023Space-Time Diffusion Features for Zero-Shot Text-Driven Motion Transfercited by 124
- 2023MagicPose: Realistic Human Poses and Facial Expressions Retargeting with Identity-aware Diffusioncited by 115
- 2023Affordance Diffusion: Synthesizing Hand-Object Interactionscited by 110
- 2025VideoPainter: Any-length Video Inpainting and Editing with Plug-and-Play Context Controlcited by 83
- 2023CCEdit: Creative and Controllable Video Editing via Diffusion Modelscited by 80
- 2023Diffusion-Guided Reconstruction of Everyday Hand-Object Interaction Clipscited by 73
- 2023Edit-A-Video: Single Video Editing with Object-Aware Consistencycited by 65
- 2024Human4DiT: 360-degree Human Video Generation with 4D Diffusion Transformercited by 61
- 2024GraspXL: Generating Grasping Motions for Diverse Objects at Scalecited by 59
- 2023Shape-aware Text-driven Layered Video Editingcited by 54
- 2025UniAnimate-DiT: Human Image Animation with Large-Scale Video Diffusion Transformercited by 40
- 2025HumanDiT: Pose-Guided Diffusion Transformer for Long-form Human Motion Video Generationcited by 38
- 2024HOIDiffusion: Generating Realistic 3D Hand-Object Interaction Datacited by 37
- 2023LEGO: Learning EGOcentric Action Frame Generation via Visual Instruction Tuningcited by 28
- 2025Stand-In: A Lightweight and Plug-and-Play Identity Control for Video Generationcited by 27
- 2024AnchorCrafter: Animate Cyber-Anchors Selling Your Products via Human-Object Interacting Video Generationcited by 25
- 2024HOI-Swap: Swapping Objects in Videos with Hand-Object Interaction Awarenesscited by 21
50 most cited of 54 All 54 (JSON)
Built on by
Citing papers per year: 2026: 1.
1 paper.