mlsanityData to 14 Oct 2026

Unified Human-Scene Interaction via Prompted Chain-of-Contacts

arXiv 2309.07918 ↗First posted 14 Sep 2023cs.CVcited by 94

Not on the map. Why?

Abstract (arXiv)

Human-Scene Interaction (HSI) is a vital component of fields like embodied AI and virtual reality. Despite advancements in motion quality and physical plausibility, two pivotal factors, versatile interaction control and the development of a user-friendly interface, require further exploration before the practical application of HSI. This paper presents a unified HSI framework, UniHSI, which supports unified control of diverse interactions through language commands. This framework is built upon the definition of interaction as Chain of Contacts (CoC): steps of human joint-object part pairs, which is inspired by the strong correlation between interaction types and human-object contact regions. Based on the definition, UniHSI constitutes a Large Language Model (LLM) Planner to translate language prompts into task plans in the form of CoC, and a Unified Controller that turns CoC into uniform task execution. To facilitate training and evaluation, we collect a new dataset named ScenePlan that encompasses thousands of task plans generated by LLMs based on diverse scenarios. Comprehensive experiments demonstrate the effectiveness of our framework in versatile task execution and generalizability to real scanned scenes. The project page is at https://github.com/OpenRobotLab/UniHSI .

Key facts. Cited by 94 papers in the map (data through 14 Oct 2026). Builds on 31 of its 47 references that are in the map. First posted on arXiv 14 Sep 2023.

Builds on

31 of 47 references.

References · 31

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2017Proximal Policy Optimization Algorithmscited by 15,215
  5. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  6. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  7. 2022Human Motion Diffusion Modelcited by 1,080
  8. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 998
  9. 2022MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Modelcited by 780
  10. 2023T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representationscited by 588
  11. 2022Executing your Commands via Motion Diffusion in Latent Spacecited by 575
  12. 2023MotionGPT: Human Motion as a Foreign Languagecited by 527
19 more
  1. 2018PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understandingcited by 526
  2. 2022MotionCLIP: Exposing Human Motion Generation to CLIP Spacecited by 408
  3. 2023Diffusion-based Generation, Optimization, and Planning in 3D Scenescited by 239
  4. 2021Robust Motion In-betweeningcited by 229
  5. 2021Stochastic Scene-Aware Motion Predictioncited by 203
  6. 2017HP-GAN: Probabilistic 3D human motion prediction via GANcited by 176
  7. 2020Populating 3D Scenes by Learning Human-Scene Interactioncited by 164
  8. 2022HUMANISE: Language-conditioned Human Motion Generation in 3D Scenescited by 158
  9. 2023MotionGPT: Finetuned LLMs Are General-Purpose Motion Generatorscited by 145
  10. 2022COUCH: Towards Controllable Human-Chair Interactionscited by 135
  11. 2018QuaterNet: A Quaternion-based Recurrent Model for Human Motioncited by 109
  12. 2023Synthesizing Diverse Human Motions in 3D Indoor Scenescited by 109
  13. 2023Synthesizing Physical Character-Scene Interactionscited by 104
  14. 2022Towards Diverse and Natural Scene-aware 3D Human Motion Synthesiscited by 91
  15. 2022Compositional Human-Scene Interaction Synthesis with Semantic Controlcited by 83
  16. 2023CIRCLE: Capture In Rich Contextual Environmentscited by 79
  17. 2023SINC: Spatial Composition of 3D Human Motions for Simultaneous Action Generationcited by 64
  18. 2023Synthesizing Physically Plausible Human Motions in 3D Scenescited by 49
  19. 2023PADL: Language-Directed Physics-Based Character Controlcited by 48

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 7; 2024: 33; 2025: 37; 2026: 17.

94 papers.

Sort:

By area:

  1. 2024MotionLCM: Real-time Controllable Motion Generation via Latent Consistency Modelcited by 168
  2. 2023CG-HOI: Contact-Guided 3D Human-Object Interaction Generationcited by 138
  3. 2024Scaling Up Dynamic Human-Scene Interaction Modelingcited by 137
  4. 2024MaskedMimic: Unified Physics-Based Character Control Through Masked Motion Inpaintingcited by 116
  5. 2023Controllable Human-Object Interaction Synthesiscited by 109
  6. 2024CLoSD: Closing the Loop between Simulation and Diffusion for multi-task character controlcited by 109
  7. 2024MotionLLM: Understanding Human Behaviors from Human Motions and Videoscited by 90
  8. 2024Move as You Say, Interact as You Can: Language-guided Human Motion Generation with Scene Affordancecited by 80
  9. 2024Generating Human Interaction Motions in Scenes with Text Controlcited by 80
  10. 2023PhysHOI: Physics-Based Imitation of Dynamic Human-Object Interactioncited by 76
  11. 2025InterMimic: Towards Universal Whole-Body Control for Physics-Based Human-Object Interactionscited by 72
  12. 2025Go to Zero: Towards Zero-shot Motion Generation with Million-scale Datacited by 68
38 more
  1. 2024Large Motion Model for Unified Multi-Modal Motion Generationcited by 65
  2. 2024InterDreamer: Zero-Shot Text to 3D Dynamic Human-Object Interactioncited by 64
  3. 2024Human-Object Interaction from Human-Level Instructionscited by 64
  4. 2023AvatarGPT: All-in-One Framework for Motion Understanding, Planning, Generation and Beyondcited by 63
  5. 2024Autonomous Character-Scene Interaction Synthesis from Text Instructioncited by 56
  6. 2024ScaMo: Exploring the Scaling Law in Autoregressive Motion Generation Modelcited by 53
  7. 2025TokenHSI: Unified Synthesis of Physical Human-Scene Interactions through Task Tokenizationcited by 51
  8. 2023LEMON: Learning 3D Human-Object Interaction Relation from 2D Imagescited by 49
  9. 2024Beyond Chain-of-Thought: A Survey of Chain-of-X Paradigms for LLMscited by 48
  10. 2024THOR: Text to Human-Object Interaction Diffusion via Relation Interventioncited by 46
  11. 2024CooHOI: Learning Cooperative Human-Object Interaction with Manipulated Object Dynamicscited by 44
  12. 2024When LLMs step into the 3D World: A Survey and Meta-Analysis of 3D Tasks via Multi-modal Large Language Modelscited by 43
  13. 2024LaserHuman: Language-guided Scene-aware Human Motion Generation in Free Environmentcited by 33
  14. 2024SkillMimic: Learning Basketball Interaction Skills from Demonstrationscited by 30
  15. 2024Unimotion: Unifying 3D Human Motion Synthesis and Understandingcited by 30
  16. 2023Plan, Posture and Go: Towards Open-World Text-to-Motion Generationcited by 26
  17. 2024ZeroHSI: Zero-Shot 4D Human-Scene Interaction by Video Generationcited by 25
  18. 2025A Survey on Human Interaction Motion Generationcited by 25
  19. 2025SkillMimic-V2: Learning Robust and Generalizable Interaction Skills from Sparse and Noisy Demonstrationscited by 25
  20. 2024F-HOI: Toward Fine-grained Semantic-Aligned 3D Human-Object Interactionscited by 24
  21. 2023InterControl: Zero-shot Human Interaction Generation by Controlling Every Jointcited by 23
  22. 2024Lodge++: High-quality and Long Dance Generation with Vivid Choreography Patternscited by 21
  23. 2025ChainHOI: Joint-based Kinematic Chain Modeling for Human-Object Interaction Generationcited by 20
  24. 2025PhysHSI: Towards a Real-World Generalizable and Natural Humanoid-Scene Interaction Systemcited by 18
  25. 2024KMM: Key Frame Mask Mamba for Extended Motion Generationcited by 16
  26. 2024Mimicking-Bench: A Benchmark for Generalizable Humanoid-Scene Interaction Learning via Human Mimickingcited by 16
  27. 2025SemGeoMo: Dynamic Contextual Human Motion Generation with Semantic and Geometric Guidancecited by 16
  28. 2024Sitcom-Crafter: A Plot-Driven Human Motion Generation System in 3D Scenescited by 14
  29. 2024SIMS: Simulating Stylized Human-Scene Interactions with Retrieval-Augmented Script Generationcited by 12
  30. 2025Human-Object Interaction via Automatically Designed VLM-Guided Motion Policycited by 12
  31. 2025CoDA: Coordinated Diffusion Noise Optimization for Whole-Body Manipulation of Articulated Objectscited by 12
  32. 2025SceneMI: Motion In-betweening for Modeling Human-Scene Interactionscited by 11
  33. 2026InterPrior: Scaling Generative Control for Physics-Based Human-Object Interactionscited by 11
  34. 2025StickMotion: Generating 3D Human Motions by Drawing a Stickmancited by 9
  35. 2024Diffusion Models are Efficient Data Generators for Human Mesh Recoverycited by 8
  36. 2025Perceiving and Acting in First-Person: A Dataset and Benchmark for Egocentric Human-Object-Human Interactionscited by 8
  37. 2024FORCE: Physics-aware Human-object Interactioncited by 7
  38. 2024HERM: Benchmarking and Enhancing Multimodal LLMs for Human-Centric Understandingcited by 7

50 most cited of 94 All 94 (JSON)