mlsanityData to 7 Oct 2026

GenHOI: Towards Object-Consistent Hand-Object Interaction with Temporally Balanced and Spatially Selective Object Injection

arXiv 2603.06048 ↗First posted 6 Mar 2026cs.CVcited by 1

Not on the map. Why?

Abstract (arXiv)

Hand-Object Interaction (HOI) remains a core challenge in digital human video synthesis, where models must generate physically plausible contact and preserve object identity across frames. Although recent HOI reenactment approaches have achieved progress, they are typically trained and evaluated in-domain and fail to generalize to complex, in-the-wild scenarios. In contrast, all-in-one video editing models exhibit broader robustness but still struggle with HOI-specific issues such as inconsistent object appearance. In this paper, we present GenHOI, a lightweight augmentation to pretrained video generation models that injects reference-object information in a temporally balanced and spatially selective manner. For temporal balancing, we propose Head-Sliding RoPE, which assigns head-specific temporal offsets to reference tokens, distributing their influence evenly across frames and mitigating the temporal decay of 3D RoPE to improve long-range object consistency. For spatial selectivity, we design a two-level spatial attention gate that concentrates object-conditioned attention on HOI regions and adaptively scales its strength, preserving background realism while enhancing interaction fidelity. Extensive qualitative and quantitative evaluations on unseen, in-the-wild scenes demonstrate that GenHOI significantly outperforms state-of-the-art HOI reenactment and all-in-one video editing methods. Project page: https://xuanhuang0.github.io/GenHOI/

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 54 of its 62 references that are in the map. First posted on arXiv 6 Mar 2026.

Builds on

54 of 62 references.

References · 54

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  6. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  7. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
  8. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  9. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
  10. 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 960
  11. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 958
  12. 2023Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animationcited by 683
38 more
  1. 2020A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wildcited by 669
  2. 2025VACE: All-in-One Video Creation and Editingcited by 467
  3. 2023FateZero: Fusing Attentions for Zero-shot Text-based Video Editingcited by 462
  4. 2023TokenFlow: Consistent Diffusion Features for Consistent Video Editingcited by 440
  5. 2020One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencingcited by 438
  6. 2024OminiControl: Minimal and Universal Control for Diffusion Transformercited by 334
  7. 2023Pix2Video: Video Editing using Image Diffusioncited by 318
  8. 2023Video-P2P: Video Editing with Cross-attention Controlcited by 312
  9. 2021Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representationcited by 288
  10. 2025Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Freecited by 266
  11. 2023Emu Video: Factorizing Text-to-Video Generation by Explicit Image Conditioningcited by 254
  12. 2023Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translationcited by 253
  13. 2024EMO: Emote Portrait Alive -- Generating Expressive Portrait Videos with Audio2Video Diffusion Model under Weak Conditionscited by 233
  14. 2024LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Controlcited by 202
  15. 2024Hallo: Hierarchical Audio-Driven Visual Synthesis for Portrait Image Animationcited by 198
  16. 2024MimicMotion: High-Quality Human Motion Video Generation with Confidence-aware Pose Guidancecited by 196
  17. 2022Latent Image Animator: Learning to Animate Images via Latent Space Navigationcited by 172
  18. 2023DreamPose: Fashion Image-to-Video Synthesis via Stable Diffusioncited by 167
  19. 2017Video Generation From Textcited by 153
  20. 2023DisCo: Disentangled Control for Realistic Human Dance Generationcited by 141
  21. 2023CG-HOI: Contact-Guided 3D Human-Object Interaction Generationcited by 137
  22. 2023Space-Time Diffusion Features for Zero-Shot Text-Driven Motion Transfercited by 124
  23. 2023MagicPose: Realistic Human Poses and Facial Expressions Retargeting with Identity-aware Diffusioncited by 115
  24. 2023Affordance Diffusion: Synthesizing Hand-Object Interactionscited by 110
  25. 2025VideoPainter: Any-length Video Inpainting and Editing with Plug-and-Play Context Controlcited by 83
  26. 2023CCEdit: Creative and Controllable Video Editing via Diffusion Modelscited by 80
  27. 2023Diffusion-Guided Reconstruction of Everyday Hand-Object Interaction Clipscited by 73
  28. 2023Edit-A-Video: Single Video Editing with Object-Aware Consistencycited by 65
  29. 2024Human4DiT: 360-degree Human Video Generation with 4D Diffusion Transformercited by 61
  30. 2024GraspXL: Generating Grasping Motions for Diverse Objects at Scalecited by 59
  31. 2023Shape-aware Text-driven Layered Video Editingcited by 54
  32. 2025UniAnimate-DiT: Human Image Animation with Large-Scale Video Diffusion Transformercited by 40
  33. 2025HumanDiT: Pose-Guided Diffusion Transformer for Long-form Human Motion Video Generationcited by 38
  34. 2024HOIDiffusion: Generating Realistic 3D Hand-Object Interaction Datacited by 37
  35. 2023LEGO: Learning EGOcentric Action Frame Generation via Visual Instruction Tuningcited by 28
  36. 2025Stand-In: A Lightweight and Plug-and-Play Identity Control for Video Generationcited by 27
  37. 2024AnchorCrafter: Animate Cyber-Anchors Selling Your Products via Human-Object Interacting Video Generationcited by 25
  38. 2024HOI-Swap: Swapping Objects in Videos with Hand-Object Interaction Awarenesscited by 21

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.