mlsanityData to 14 Oct 2026

Harnessing Coupled Stream Completion For Human-Object Interaction Modeling

arXiv 2609.32551 ↗First posted 26 Sep 2026cs.CV cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Text-conditioned human-object interaction (HOI) generation requires body motion, object trajectories & rotations, and hand articulation to remain coordinated. These components differ in scale and dynamics, but must agree on contact, relative pose, and timing. A shared representation may limit the distinct structure of each stream, while independent generation prevents each stream from responding to changes in the others. Latent supervision alone also does not directly constrain contact after decoding. We propose TRACE, a continuous latent framework that keeps stream states separate and couples their updates. TRACE encodes body, object, and hand motion into separate latents and predicts each stream velocity from the complete current interaction state. Geometric losses on decoded motion further constrain contact and object-relative motion over time. The same model supports completion of any single absent stream from the other two. Frozen flow features also serve as input to a language model for HOI understanding. Experiments on InterAct, OMOMO, and BEHAVE show that joint completion training improves generation and that frozen flow features improve understanding over raw-motion encoding. On InterAct, TRACE achieves the highest contact precision, recall, and F1 among the compared methods.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 38 of its 46 references that are in the map. First posted on arXiv 26 Sep 2026.

Builds on

38 of 46 references.

References · 38

  1. 2022Scalable Diffusion Models with Transformerscited by 6,054
  2. 2022Flow Matching for Generative Modelingcited by 5,702
  3. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  4. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  5. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  6. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,612
  7. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,214
  8. 2025Large Language Diffusion Modelscited by 912
  9. 2022Executing your Commands via Motion Diffusion in Latent Spacecited by 575
  10. 2023MotionGPT: Human Motion as a Foreign Languagecited by 527
  11. 2022TM2T: Stochastic and Tokenized Modeling for the Reciprocal Generation of 3D Human Motions and Textscited by 359
  12. 2022BEHAVE: Dataset and Method for Tracking Human Object Interactionscited by 249
26 more
  1. 2023Object Motion Guided Human Motion Synthesiscited by 212
  2. 2023InterDiff: Generating 3D Human-Object Interactions with Physics-Informed Diffusioncited by 182
  3. 2025LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuningcited by 148
  4. 2023CG-HOI: Contact-Guided 3D Human-Object Interaction Generationcited by 138
  5. 2023Controllable Human-Object Interaction Synthesiscited by 109
  6. 2019Efficient Learning on Point Clouds with Basis Point Setscited by 103
  7. 2023HOI-Diff: Text-Driven Synthesis of 3D Human-Object Interactions using Diffusion Modelscited by 93
  8. 2024Text2HOI: Text-guided 3D Motion Generation for Hand-Object Interactioncited by 86
  9. 2024InterDreamer: Zero-Shot Text to 3D Dynamic Human-Object Interactioncited by 64
  10. 2024Human-Object Interaction from Human-Level Instructionscited by 64
  11. 2024LaMP: Language-Motion Pretraining for Motion Generation, Retrieval, and Captioningcited by 59
  12. 2024THOR: Text to Human-Object Interaction Diffusion via Relation Interventioncited by 46
  13. 2024OmniFlow: Any-to-Any Generation with Multi-Modal Rectified Flowscited by 46
  14. 2025MotionGPT3: Human Motion as a Second Modalitycited by 37
  15. 2025HOIGPT: Learning Long Sequence Hand-Object Interaction with Language Modelscited by 33
  16. 2025InterAct: Advancing Large-Scale Versatile 3D Human-Object Interaction Generationcited by 32
  17. 2025Guiding Human-Object Interactions with Rich Geometry and Relationscited by 24
  18. 2025InteractVLM: 3D Interaction Reasoning from 2D Foundational Modelscited by 21
  19. 2024TriDi: Trilateral Diffusion of 3D Humans, Objects, and Interactionscited by 20
  20. 2025ChainHOI: Joint-based Kinematic Chain Modeling for Human-Object Interaction Generationcited by 20
  21. 2024Diffgrasp: Whole-Body Grasping Synthesis Guided by Object Motion Using a Diffusion Modelcited by 13
  22. 2025Auto-Regressive Diffusion for Generating 3D Human-Object Interactionscited by 12
  23. 2024OOD-HOI: Text-Driven 3D Whole-Body Human-Object Interactions Generation Beyond Training Domainscited by 9
  24. 2026Unleashing Guidance Without Classifiers for Human-Object Interaction Animationcited by 6
  25. 2026Uni-HOI:A Unified framework for Learning the Joint distribution of Text and Human-Object Interactioncited by 2
  26. 2026FlowCoMotion: Text-to-Motion Generation via Token-Latent Flow Modelingcited by 1

Built on by

Not cited yet.