TeamHOI: Learning a Unified Policy for Cooperative Human-Object Interactions with Any Team Size
Not on the map. Why?
Abstract (arXiv)
Physics-based humanoid control has achieved remarkable progress in enabling realistic and high-performing single-agent behaviors, yet extending these capabilities to cooperative human-object interaction (HOI) remains challenging. We present TeamHOI, a framework that enables a single decentralized policy to handle cooperative HOIs across any number of cooperating agents. Each agent operates using local observations while attending to other teammates through a Transformer-based policy network with teammate tokens, allowing scalable coordination across variable team sizes. To enforce motion realism while addressing the scarcity of cooperative HOI data, we further introduce a masked Adversarial Motion Prior (AMP) strategy that uses single-human reference motions while masking object-interacting body parts during training. The masked regions are then guided through task rewards to produce diverse and physically plausible cooperative behaviors. We evaluate TeamHOI on a challenging cooperative carrying task involving two to eight humanoid agents and varied object geometries. Finally, to promote stable carrying, we design a team-size- and shape-agnostic formation reward. TeamHOI achieves high success rates and demonstrates coherent cooperation across diverse configurations with a single policy.
Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 58 of its 78 references that are in the map. First posted on arXiv 9 Mar 2026.
Builds on
58 of 78 references.
References · 58
- 2017Attention Is All You Needcited by 60,361
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,236
- 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 998
- 2023Human Motion Diffusion as a Generative Priorcited by 357
- 2023Guided Motion Diffusion for Controllable Human Motion Synthesiscited by 223
- 2023Perpetual Humanoid Control for Real-time Simulated Avatarscited by 218
- 2023InterGen: Diffusion-based Multi-human Motion Generation under Complex Interactionscited by 206
- 2023Trace and Pace: Controllable Pedestrian Animation via Guided Trajectory Diffusioncited by 138
- 2023Universal Humanoid Motion Representations for Physics-Based Controlcited by 116
- 2024MaskedMimic: Unified Physics-Based Character Control Through Masked Motion Inpaintingcited by 116
- 2024CLoSD: Closing the Loop between Simulation and Diffusion for multi-task character controlcited by 109
38 more
- 2023Inter-X: Towards Versatile Human-Human Interaction Analysiscited by 98
- 2023CALM: Conditional Adversarial Latent Models for Directable Virtual Characterscited by 94
- 2023Unified Human-Scene Interaction via Prompted Chain-of-Contactscited by 94
- 2024HOVER: Versatile Neural Whole-Body Controller for Humanoid Robotscited by 91
- 2025InterMimic: Towards Universal Whole-Body Control for Physics-Based Human-Object Interactionscited by 72
- 2024Human-Object Interaction from Human-Level Instructionscited by 64
- 2023ReMoS: 3D Motion-Conditioned Reaction Synthesis for Two-Person Interactionscited by 62
- 2025Humanoid Locomotion and Manipulation: Current Progress and Challenges in Control, Planning, and Learningcited by 61
- 2023C·ASE: Learning Conditional Adversarial Skill Embeddings for Physics-based Characterscited by 55
- 2024HOI-M3:Capture Multiple Humans and Objects Interaction within Contextual Environmentcited by 53
- 2023Physically Plausible Full-Body Hand-Object Interaction Synthesiscited by 52
- 2025TokenHSI: Unified Synthesis of Physical Human-Scene Interactions through Task Tokenizationcited by 51
- 2023TEDi: Temporally-Entangled Diffusion for Long-Term Motion Synthesiscited by 49
- 2023Synthesizing Physically Plausible Human Motions in 3D Scenescited by 49
- 2023PADL: Language-Directed Physics-Based Character Controlcited by 48
- 2024Duolando: Follower GPT with Off-Policy Reinforcement Learning for Dance Accompanimentcited by 47
- 2024InterMask: 3D Human Interaction Generation via Collaborative Masked Modelingcited by 47
- 2024Omnigrasp: Grasping Diverse Objects with Simulated Humanoidscited by 46
- 2023Hierarchical Planning and Control for Box Loco-Manipulationcited by 44
- 2024CooHOI: Learning Cooperative Human-Object Interaction with Manipulated Object Dynamicscited by 44
- 2024CORE4D: A 4D Human-Object-Human Interaction Dataset for Collaborative Object REarrangementcited by 44
- 2024FreeMotion: A Unified Framework for Number-free Text-to-Motion Synthesiscited by 42
- 2019Learning to Sit: Synthesizing Human-Chair Interactions via Hierarchical Controlcited by 40
- 2023PMP: Learning to Physically Interact with Environments using Part-wise Motion Priorscited by 38
- 2023Music-Driven Group Choreographycited by 37
- 2023Simulation and Retargeting of Complex Multi-Character Interactionscited by 34
- 2024SkillMimic: Learning Basketball Interaction Skills from Demonstrationscited by 30
- 2024SMPLOlympics: Sports Environments for Physically Simulated Humanoidscited by 29
- 2025Diffuse-CLoC: Guided Diffusion for Physics-based Character Look-ahead Controlcited by 28
- 2025A Survey on Human Interaction Motion Generationcited by 25
- 2025UniPhys: Unified Planner and Controller with Diffusion for Flexible Physics-Based Character Controlcited by 25
- 2025SkillMimic-V2: Learning Robust and Generalizable Interaction Skills from Sparse and Noisy Demonstrationscited by 25
- 2025DuetGen: Music Driven Two-Person Dance Generation via Hierarchical Masked Modelingcited by 21
- 2024Strategy and Skill Learning for Physics-based Table Tennis Animationcited by 20
- 2025Towards Immersive Human-X Interaction: A Real-Time Framework for Physically Plausible Motion Synthesiscited by 20
- 2024PhysReaction: Physically Plausible Real-Time Humanoid Reaction Synthesis via Forward Dynamics Guided 4D Imitationcited by 18
- 2025PhysHSI: Towards a Real-World Generalizable and Natural Humanoid-Scene Interaction Systemcited by 18
- 2024Mimicking-Bench: A Benchmark for Generalizable Humanoid-Scene Interaction Learning via Human Mimickingcited by 16
50 most cited of 58 All 58 (JSON)
Built on by
Citing papers per year: 2026: 5.
5 papers.
- 2026Learning Multi-Humanoid Pickup and Transport via Decentralized Object-Centric Controlcited by 0
- 2026CoHuB: A Simulation Benchmark for Multi-Humanoid Collaborationcited by 0
- 2026MASkillBlender: Decentralized Whole-Body Coordination for Multi-Humanoid Loco-Manipulation via Skill Blendingcited by 0
- 2026From Digital Human Interactions to Physics-Based Humanoid Skills: Physics-Grounded Post-Training of Interaction Generatorscited by 0
- 2026Scalable Multi-Human Cooperative Manipulation via Commander-Executor Coordinationcited by 0
- 2026Scalable Multi-Human Cooperative Manipulation via Commander-Executor Coordination10 Oct 2026
- 2026From Digital Human Interactions to Physics-Based Humanoid Skills: Physics-Grounded Post-Training of Interaction Generators7 Oct 2026
- 2026MASkillBlender: Decentralized Whole-Body Coordination for Multi-Humanoid Loco-Manipulation via Skill Blending1 Oct 2026
- 2026CoHuB: A Simulation Benchmark for Multi-Humanoid Collaboration28 Sep 2026
- 2026Learning Multi-Humanoid Pickup and Transport via Decentralized Object-Centric Control15 Sep 2026