mlsanityData to 7 Oct 2026

PhysHOI: Physics-Based Imitation of Dynamic Human-Object Interaction

arXiv 2312.04393 ↗First posted 7 Dec 2023cs.CV cs.GR cs.ROcited by 72

Not on the map. Why?

Abstract (arXiv)

Humans interact with objects all the time. Enabling a humanoid to learn human-object interaction (HOI) is a key step for future smart animation and intelligent robotics systems. However, recent progress in physics-based HOI requires carefully designed task-specific rewards, making the system unscalable and labor-intensive. This work focuses on dynamic HOI imitation: teaching humanoid dynamic interaction skills through imitating kinematic HOI demonstrations. It is quite challenging because of the complexity of the interaction between body parts and objects and the lack of dynamic HOI data. To handle the above issues, we present PhysHOI, the first physics-based whole-body HOI imitation approach without task-specific reward designs. Except for the kinematic HOI representations of humans and objects, we introduce the contact graph to model the contact relations between body parts and objects explicitly. A contact graph reward is also designed, which proved to be critical for precise HOI imitation. Based on the key designs, PhysHOI can imitate diverse HOI tasks simply yet effectively without prior knowledge. To make up for the lack of dynamic HOI scenarios in this area, we introduce the BallPlay dataset that contains eight whole-body basketball skills. We validate PhysHOI on diverse HOI tasks, including whole-body grasping and basketball skills.

Key facts. Cited by 72 papers in the map (data through 7 Oct 2026). Builds on 87 of its 121 references that are in the map. First posted on arXiv 7 Dec 2023.

Builds on

87 of 121 references.

References · 87

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2020Denoising Diffusion Implicit Modelscited by 9,655
  5. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  6. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  7. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  8. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,600
  9. 2016Generative Adversarial Imitation Learningcited by 1,576
  10. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,208
  11. 2022Human Motion Diffusion Modelcited by 1,062
  12. 2022Denoising Diffusion Restoration Modelscited by 951
38 more
  1. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 882
  2. 2022MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Modelcited by 768
  3. 2023ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depthcited by 646
  4. 2023T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representationscited by 577
  5. 2022Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Modelcited by 574
  6. 2022Executing your Commands via Motion Diffusion in Latent Spacecited by 567
  7. 2021Action-Conditioned 3D Human Motion Synthesis with Transformer VAEcited by 497
  8. 2020Action2Motion: Conditioned Generation of 3D Human Motionscited by 438
  9. 2020GRAB: A Dataset of Whole-Body Human Grasping of Objectscited by 397
  10. 2017On human motion prediction using recurrent neural networkscited by 336
  11. 2016The KIT Motion-Language Datasetcited by 334
  12. 2022ARCTIC: A Dataset for Dexterous Bimanual Hand-Object Manipulationcited by 306
  13. 2022EDGE: Editable Dance Generation From Musiccited by 284
  14. 2022BEHAVE: Dataset and Method for Tracking Human Object Interactionscited by 246
  15. 2020Learning Agile Robotic Locomotion Skills by Imitating Animalscited by 229
  16. 2021Robust Motion In-betweeningcited by 223
  17. 2023Motion-X: A Large-scale 3D Expressive Whole-body Human Motion Datasetcited by 220
  18. 2022MoFusion: A Framework for Denoising-Diffusion-based Motion Synthesiscited by 218
  19. 2023Object Motion Guided Human Motion Synthesiscited by 206
  20. 2021Stochastic Scene-Aware Motion Predictioncited by 203
  21. 2017Detecting and Recognizing Human-Object Interactionscited by 199
  22. 2023Perpetual Humanoid Control for Real-time Simulated Avatarscited by 198
  23. 2021Character Controllers Using Motion VAEscited by 182
  24. 2023InterDiff: Generating 3D Human-Object Interactions with Physics-Informed Diffusioncited by 181
  25. 2022Human-to-Robot Imitation in the Wildcited by 157
  26. 2018DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skillscited by 153
  27. 2020Perceiving 3D Human-Object Spatial Arrangements from a Single Image in the Wildcited by 152
  28. 2021GOAL: Generating 4D Whole-Body Motion for Hand-Object Graspingcited by 142
  29. 2022GAPartNet: Cross-Category Domain-Generalizable Object Perception and Manipulation via Generalizable and Actionable Partscited by 135
  30. 2022COUCH: Towards Controllable Human-Chair Interactionscited by 134
  31. 2023Taming Diffusion Models for Audio-Driven Co-Speech Gesture Generationcited by 128
  32. 2022IMos: Intent-Driven Full-Body Motion Synthesis for Human-Object Interactionscited by 121
  33. 2023UniDexGrasp: Universal Robotic Dexterous Grasping via Learning Diverse Proposal Generation and Goal-Conditioned Policycited by 117
  34. 2018Robots Learn Social Skills: End-to-End Learning of Co-Speech Gesture Generation for Humanoid Robotscited by 110
  35. 2021D-Grasp: Physically Plausible Dynamic Grasp Synthesis for Hand-Object Interactionscited by 109
  36. 2023HumanTOMATO: Text-aligned Whole-body Motion Generationcited by 103
  37. 2023Synthesizing Physical Character-Scene Interactionscited by 102
  38. 2021Dynamics-Regulated Kinematic Policy for Egocentric Pose Estimationcited by 101

50 most cited of 87 All 87 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 18; 2025: 36; 2026: 17.

72 papers.

Sort:

By area:

  1. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 914
  2. 2025ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body Skillscited by 104
  3. 2024HumanoidBench: Simulated Humanoid Benchmark for Whole-Body Locomotion and Manipulationcited by 101
  4. 2024MaskedMimic: Unified Physics-Based Character Control Through Masked Motion Inpaintingcited by 101
  5. 2023HOI-Diff: Text-Driven Synthesis of 3D Human-Object Interactions using Diffusion Modelscited by 89
  6. 2025InterMimic: Towards Universal Whole-Body Control for Physics-Based Human-Object Interactionscited by 67
  7. 2024Large Motion Model for Unified Multi-Modal Motion Generationcited by 65
  8. 2024InterDreamer: Zero-Shot Text to 3D Dynamic Human-Object Interactioncited by 63
  9. 2024Human-Object Interaction from Human-Level Instructionscited by 62
  10. 2024ParaHome: Parameterizing Everyday Home Activities Towards 3D Generative Modeling of Human-Object Interactionscited by 54
  11. 2024HOI-M3:Capture Multiple Humans and Objects Interaction within Contextual Environmentcited by 52
  12. 2025Zero-Shot Whole-Body Humanoid Control via Behavioral Foundation Modelscited by 51
38 more
  1. 2024CORE4D: A 4D Human-Object-Human Interaction Dataset for Collaborative Object REarrangementcited by 40
  2. 2025InterAct: Advancing Large-Scale Versatile 3D Human-Object Interaction Generationcited by 31
  3. 2024SMPLOlympics: Sports Environments for Physically Simulated Humanoidscited by 29
  4. 2024SkillMimic: Learning Basketball Interaction Skills from Demonstrationscited by 25
  5. 2025A Survey on Human Interaction Motion Generationcited by 25
  6. 2024F-HOI: Toward Fine-grained Semantic-Aligned 3D Human-Object Interactionscited by 24
  7. 2025SkillMimic-V2: Learning Robust and Generalizable Interaction Skills from Sparse and Noisy Demonstrationscited by 23
  8. 2024Mimicking-Bench: A Benchmark for Generalizable Humanoid-Scene Interaction Learning via Human Mimickingcited by 14
  9. 2025DexTrack: Towards Generalizable Neural Tracking Control for Dexterous Manipulation from Human Referencescited by 13
  10. 2025Reconstructing 4D Spatial Intelligence: A Surveycited by 13
  11. 2025Auto-Regressive Diffusion for Generating 3D Human-Object Interactionscited by 12
  12. 2025SViMo: Synchronized Diffusion for Video and Motion Generation in Hand-object Interaction Scenarioscited by 11
  13. 2024QuasiSim: Parameterized Quasi-Physical Simulators for Dexterous Manipulations Transfercited by 10
  14. 2025Human-Object Interaction via Automatically Designed VLM-Guided Motion Policycited by 10
  15. 2025CoDA: Coordinated Diffusion Noise Optimization for Whole-Body Manipulation of Articulated Objectscited by 10
  16. 2025HOI-Dyn: Learning Interaction Dynamics for Human-Object Motion Diffusioncited by 10
  17. 2026HumanX: Toward Agile and Generalizable Humanoid Interaction Skills from Human Videoscited by 10
  18. 2026InterPrior: Scaling Generative Control for Physics-Based Human-Object Interactionscited by 10
  19. 2025DyTact: Capturing Dynamic Contacts in Hand-Object Manipulationcited by 8
  20. 2024FORCE: Physics-aware Human-object Interactioncited by 7
  21. 2025Target-Aware Video Diffusion Modelscited by 7
  22. 2025Zero-Shot Human-Object Interaction Synthesis with Multimodal Priorscited by 7
  23. 2025GenHOI: Generalizing Text-driven 4D Human-Object Interaction Synthesis for Unseen Objectscited by 7
  24. 20253D Human Interaction Generation: A Surveycited by 6
  25. 2025Learning to Ball: Composing Policies for Long-Horizon Basketball Movescited by 6
  26. 2026FlowHOI: Flow-based Semantics-Grounded Generation of Hand-Object Interactions for Dexterous Robot Manipulationcited by 6
  27. 2025Emergent Active Perception and Dexterity of Simulated Humanoids from Visual Reinforcement Learningcited by 5
  28. 2025HOSIG: Full-Body Human-Object-Scene Interaction Generation with Hierarchical Scene Perceptioncited by 5
  29. 2025DAViD: Modeling Dynamic Affordance of 3D Objects Using Pre-trained Video Diffusion Modelscited by 4
  30. 2025HOI-PAGE: Zero-Shot Human-Object Interaction Generation with Part Affordance Guidancecited by 4
  31. 2025PhysiInter: Integrating Physical Mapping for High-Fidelity Human Interaction Generationcited by 4
  32. 2025Half-Physics: Enabling Kinematic 3D Human Model with Physical Interactionscited by 4
  33. 2025Versatile Physics-based Character Control with Hybrid Latent Representationcited by 3
  34. 2025Humanoid Agent via Embodied Chain-of-Action Reasoning with Multimodal Foundation Models for Zero-Shot Loco-Manipulationcited by 3
  35. 2025SimGenHOI: Physically Realistic Whole-Body Humanoid-Object Interaction via Generative Modeling and Reinforcement Learningcited by 3
  36. 2025Efficient and Scalable Monocular Human-Object Interaction Motion Reconstructioncited by 3
  37. 2026SafeFlow: Real-Time Text-Driven Humanoid Whole-Body Control via Physics-Guided Rectified Flow and Selective Safety Gatingcited by 3
  38. 2026HarmoHOI: Harmonizing Appearance and 3D Motion for Multi-view Hand-Object Interaction Synthesiscited by 3

50 most cited of 72 All 72 (JSON)