mlsanityData to 14 Oct 2026

Towards Diverse and Natural Scene-aware 3D Human Motion Synthesis

arXiv 2205.13001 ↗First posted 25 May 2022cs.CVcited by 91

Not on the map. Why?

Abstract (arXiv)

The ability to synthesize long-term human motion sequences in real-world scenes can facilitate numerous applications. Previous approaches for scene-aware motion synthesis are constrained by pre-defined target objects or positions and thus limit the diversity of human-scene interactions for synthesized motions. In this paper, we focus on the problem of synthesizing diverse scene-aware human motions under the guidance of target action sequences. To achieve this, we first decompose the diversity of scene-aware human motions into three aspects, namely interaction diversity (e.g. sitting on different objects with different poses in the given scenes), path diversity (e.g. moving to the target locations following different paths), and the motion diversity (e.g. having various body movements during moving). Based on this factorized scheme, a hierarchical framework is proposed, with each sub-module responsible for modeling one aspect. We assess the effectiveness of our framework on two challenging datasets for scene-aware human motion synthesis. The experiment results show that the proposed framework remarkably outperforms previous methods in terms of diversity and naturalness.

Key facts. Cited by 91 papers in the map (data through 14 Oct 2026). Builds on 30 of its 44 references that are in the map. First posted on arXiv 25 May 2022.

Builds on

30 of 44 references.

References · 30

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  4. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  5. 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
  6. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,612
  7. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
  8. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,214
  9. 2018Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognitioncited by 1,209
  10. 2021Action-Conditioned 3D Human Motion Synthesis with Transformer VAEcited by 501
  11. 2020Action2Motion: Conditioned Generation of 3D Human Motionscited by 441
  12. 2020GRAB: A Dataset of Whole-Body Human Grasping of Objectscited by 405
18 more
  1. 2017On human motion prediction using recurrent neural networkscited by 338
  2. 2019Resolving 3D Human Pose Ambiguities with 3D Scene Constraintscited by 295
  3. 2019Learning Trajectory Dependencies for Human Motion Predictioncited by 246
  4. 2021Robust Motion In-betweeningcited by 229
  5. 2021Stochastic Scene-Aware Motion Predictioncited by 203
  6. 2020Long-term Human Motion Prediction with Scene Contextcited by 192
  7. 2020DLow: Diversifying Latent Flows for Diverse Human Motion Predictioncited by 185
  8. 2017HP-GAN: Probabilistic 3D human motion prediction via GANcited by 176
  9. 2020Populating 3D Scenes by Learning Human-Scene Interactioncited by 164
  10. 2020Synthesizing Long-Term 3D Human Motion and Interaction in 3D Scenescited by 135
  11. 2020Dynamic Multiscale Graph Neural Networks for 3D Skeleton-Based Human Motion Predictioncited by 132
  12. 2019Generating 3D People in Scenes without Peoplecited by 127
  13. 2020We are More than Our Joints: Predicting how 3D Bodies Movecited by 115
  14. 2018QuaterNet: A Quaternion-based Recurrent Model for Human Motioncited by 109
  15. 2021Scene-aware Generative Network for Human Motion Synthesiscited by 70
  16. 2018Pose Guided Human Video Generationcited by 67
  17. 2017Deep Video Generation, Prediction and Completion of Human Action Sequencescited by 52
  18. 2020Hierarchical Style-based Networks for Motion Synthesiscited by 23

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 4; 2023: 29; 2024: 30; 2025: 22; 2026: 6.

91 papers.

Sort:

By area:

  1. 2023Diffusion-based Generation, Optimization, and Planning in 3D Scenescited by 239
  2. 2023OmniControl: Control Any Joint at Any Time for Human Motion Generationcited by 238
  3. 2023InterDiff: Generating 3D Human-Object Interactions with Physics-Informed Diffusioncited by 182
  4. 2023CG-HOI: Contact-Guided 3D Human-Object Interaction Generationcited by 138
  5. 2024Scaling Up Dynamic Human-Scene Interaction Modelingcited by 137
  6. 2024MaskedMimic: Unified Physics-Based Character Control Through Masked Motion Inpaintingcited by 116
  7. 2023Synthesizing Diverse Human Motions in 3D Indoor Scenescited by 109
  8. 2023Controllable Human-Object Interaction Synthesiscited by 109
  9. 2023Synthesizing Physical Character-Scene Interactionscited by 104
  10. 2023HumanTOMATO: Text-aligned Whole-body Motion Generationcited by 104
  11. 2023Human Motion Generation: A Surveycited by 96
  12. 2023Unified Human-Scene Interaction via Prompted Chain-of-Contactscited by 94
38 more
  1. 2023HOI-Diff: Text-Driven Synthesis of 3D Human-Object Interactions using Diffusion Modelscited by 93
  2. 2024MotionLLM: Understanding Human Behaviors from Human Motions and Videoscited by 90
  3. 2024Rethinking Diffusion for Text-Driven Human Motion Generation: Redundant Representations, Evaluation, and Masked Autoregressioncited by 85
  4. 2022Compositional Human-Scene Interaction Synthesis with Semantic Controlcited by 83
  5. 2024Seamless Human Motion Composition with Blended Positional Encodingscited by 80
  6. 2024Move as You Say, Interact as You Can: Language-guided Human Motion Generation with Scene Affordancecited by 80
  7. 2023NIFTY: Neural Object Interaction Fields for Guided Human Motion Synthesiscited by 78
  8. 2025InterMimic: Towards Universal Whole-Body Control for Physics-Based Human-Object Interactionscited by 72
  9. 2023SINC: Spatial Composition of 3D Human Motions for Simultaneous Action Generationcited by 64
  10. 2024InterDreamer: Zero-Shot Text to 3D Dynamic Human-Object Interactioncited by 64
  11. 2024Human-Object Interaction from Human-Level Instructionscited by 64
  12. 2024ParaHome: Parameterizing Everyday Home Activities Towards 3D Generative Modeling of Human-Object Interactionscited by 57
  13. 2023Interactive Character Control with Auto-Regressive Motion Diffusion Modelscited by 55
  14. 2023C·ASE: Learning Conditional Adversarial Skill Embeddings for Physics-based Characterscited by 55
  15. 2023Locomotion-Action-Manipulation: Synthesizing Human-Scene Interactions in Complex 3D Environmentscited by 51
  16. 2023Synthesizing Physically Plausible Human Motions in 3D Scenescited by 49
  17. 2024Generating Human Motion in 3D Scenes from Text Descriptionscited by 49
  18. 2023Hierarchical Generation of Human-Object Interactions with Diffusion Probabilistic Modelscited by 43
  19. 2023Task-Oriented Human-Object Interactions Generation with Implicit Neural Representationscited by 42
  20. 2022TORE: Token Reduction for Efficient Human Mesh Recovery with Transformercited by 41
  21. 2023GenZI: Zero-Shot 3D Human-Scene Interaction Generationcited by 40
  22. 2023Generating Continual Human Motion in Diverse 3D Scenescited by 39
  23. 2023PMP: Learning to Physically Interact with Environments using Part-wise Motion Priorscited by 38
  24. 2024Unimotion: Unifying 3D Human Motion Synthesis and Understandingcited by 30
  25. 2025Absolute Coordinates Make Motion Generation Easycited by 29
  26. 2024HMD^2: Environment-aware Motion Generation from Single Egocentric Head-Mounted Devicecited by 27
  27. 2023Revisit Human-Scene Interaction via Space Occupancycited by 25
  28. 2024ZeroHSI: Zero-Shot 4D Human-Scene Interaction by Video Generationcited by 25
  29. 2025A Survey on Human Interaction Motion Generationcited by 25
  30. 20253D Scene Generation: A Surveycited by 24
  31. 2024GEARS: Local Geometry-aware Hand-object Interaction Synthesiscited by 20
  32. 2024Programmable Motion Generation for Open-Set Motion Control Taskscited by 20
  33. 2024A Unified Diffusion Framework for Scene-aware Human Motion Estimation from Sparse Signalscited by 16
  34. 2023ImitationNet: Unsupervised Human-to-Robot Motion Retargeting via Shared Latent Spacecited by 12
  35. 2024SIMS: Simulating Stylized Human-Scene Interactions with Retrieval-Augmented Script Generationcited by 12
  36. 2025Human-Object Interaction via Automatically Designed VLM-Guided Motion Policycited by 12
  37. 2025CoDA: Coordinated Diffusion Noise Optimization for Whole-Body Manipulation of Articulated Objectscited by 12
  38. 2025MOSPA: Human Motion Generation Driven by Spatial Audiocited by 12

50 most cited of 91 All 91 (JSON)