mlsanityData to 7 Oct 2026

Scene Synthesis from Human Motion

arXiv 2301.01424 ↗First posted 4 Jan 2023cs.GR cs.CVcited by 20

Not on the map. Why?

Abstract (arXiv)

Large-scale capture of human motion with diverse, complex scenes, while immensely useful, is often considered prohibitively costly. Meanwhile, human motion alone contains rich information about the scene they reside in and interact with. For example, a sitting human suggests the existence of a chair, and their leg position further implies the chair's pose. In this paper, we propose to synthesize diverse, semantically reasonable, and physically plausible scenes based on human motion. Our framework, Scene Synthesis from HUMan MotiON (SUMMON), includes two steps. It first uses ContactFormer, our newly introduced contact predictor, to obtain temporally consistent contact labels from human motion. Based on these predictions, SUMMON then chooses interacting objects and optimizes physical plausibility losses; it further populates the scene with objects that do not interact with humans. Experimental results demonstrate that SUMMON synthesizes feasible, plausible, and diverse scenes and has the potential to generate extensive human-scene interaction data for the community.

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Builds on 33 of its 51 references that are in the map. First posted on arXiv 4 Jan 2023.

Builds on

33 of 51 references.

References · 33

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,600
  3. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,208
  4. 2016Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Imagecited by 692
  5. 2018VirtualHome: Simulating Household Activities via Programscited by 411
  6. 20203D-FRONT: 3D Furnished Rooms with layOuts and semaNTicscited by 375
  7. 2021HuMoR: 3D Human Motion Model for Robust Pose Estimationcited by 319
  8. 2015LSTM: A Search Space Odysseycited by 315
  9. 20203D-FUTURE: 3D Furniture shape with TextUREcited by 306
  10. 2019Resolving 3D Human Pose Ambiguities with 3D Scene Constraintscited by 294
  11. 2022BEHAVE: Dataset and Method for Tracking Human Object Interactionscited by 246
  12. 2021Robust Motion In-betweeningcited by 223
21 more
  1. 2021Stochastic Scene-Aware Motion Predictioncited by 203
  2. 2021ATISS: Autoregressive Transformers for Indoor Scene Synthesiscited by 195
  3. 2020Populating 3D Scenes by Learning Human-Scene Interactioncited by 164
  4. 2020SceneFormer: Indoor Scene Generation with Transformerscited by 154
  5. 2020Synthesizing Long-Term 3D Human Motion and Interaction in 3D Scenescited by 135
  6. 2019Generating 3D People in Scenes without Peoplecited by 127
  7. 2018Fast and Flexible Indoor Scene Synthesis via Deep Convolutional Generative Modelscited by 105
  8. 2021Learning Motion Priors for 4D Human Body Capture in 3D Scenescited by 98
  9. 2019SpiralNet++: A Fast and Highly Efficient Mesh Convolution Operatorcited by 89
  10. 2022GIMO: Gaze-Informed Human Motion Prediction in Contextcited by 81
  11. 2017Learning to Act Properly: Predicting and Explaining Affordances from Imagescited by 74
  12. 2019Putting Humans in a Scene: Learning Affordance in 3D Indoor Environmentscited by 72
  13. 2019Holistic++ Scene Understanding: Single-view 3D Holistic Scene Parsing and Human Pose Estimation with Human-Object Interaction and Physical Commonsensecited by 71
  14. 2022Human-Aware Object Placement for Visual Environment Reconstructioncited by 56
  15. 2019SceneGraphNet: Neural Message Passing for 3D Indoor Scene Augmentationcited by 55
  16. 2020End-to-End Optimization of Scene Layoutcited by 52
  17. 2018Binge Watching: Scaling Affordance Learning from Sitcomscited by 41
  18. 2020Hierarchical Style-based Networks for Motion Synthesiscited by 23
  19. 2021Pose2Room: Understanding 3D Scenes from Human Activitiescited by 22
  20. 2020Fast 3D Indoor Scene Synthesis with Discrete and Exact Layout Pattern Extractioncited by 7
  21. 2021Learning Body-Aware 3D Shape Generative Modelscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 7; 2024: 7; 2025: 3; 2026: 3.

20 papers.

Sort:

By area:

  1. 2024Generating Human Interaction Motions in Scenes with Text Controlcited by 80
  2. 2023NIFTY: Neural Object Interaction Fields for Guided Human Motion Synthesiscited by 78
  3. 2023Diffusion-Guided Reconstruction of Everyday Hand-Object Interaction Clipscited by 73
  4. 2024ParaHome: Parameterizing Everyday Home Activities Towards 3D Generative Modeling of Human-Object Interactionscited by 54
  5. 2023LEMON: Learning 3D Human-Object Interaction Relation from 2D Imagescited by 49
  6. 2024THOR: Text to Human-Object Interaction Diffusion via Relation Interventioncited by 46
  7. 2023Revisit Human-Scene Interaction via Space Occupancycited by 24
  8. 20253D Scene Generation: A Surveycited by 24
  9. 2023ROAM: Robust and Object-Aware Motion Generation Using Neural Pose Descriptorscited by 23
  10. 2024TriDi: Trilateral Diffusion of 3D Humans, Objects, and Interactionscited by 18
  11. 2023Language-driven Scene Synthesis using Multi-conditional Diffusion Modelcited by 10
  12. 2024Physics-based Scene Layout Generation from Human Motioncited by 10
8 more
  1. 2023Ins-HOI: Instance Aware Human-Object Interactions Recoverycited by 7
  2. 2025ReSpace: Text-Driven Autoregressive 3D Indoor Scene Synthesis and Editingcited by 5
  3. 2024CRAFT: Designing Creative and Functional 3D Objectscited by 2
  4. 2024Diverse 3D Human Pose Generation in Scenes based on Decoupled Structurecited by 1
  5. 2025AccioScene: Compositional 3D Scene Generation via Graph Diffusion and Interaction-driven Criticscited by 1
  6. 2026Seeing Without Eyes: 4D Human-Scene Understanding from Wearable IMUscited by 1
  7. 2026SceneTeract: Probing and Improving Agent-Aware Activity Reasoning in 3D Indoor Scenescited by 0
  8. 2026Efficient Human-Contact Representation for Human-Scene Interactioncited by 0