mlsanityData to 7 Oct 2026

Stochastic Scene-Aware Motion Prediction

arXiv 2108.08284 ↗First posted 18 Aug 2021cs.CVcited by 203

Not on the map. Why?

Cited by 22 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

A long-standing goal in computer vision is to capture, model, and realistically synthesize human behavior. Specifically, by learning from data, our goal is to enable virtual humans to navigate within cluttered indoor scenes and naturally interact with objects. Such embodied behavior has applications in virtual reality, computer games, and robotics, while synthesized behavior can be used as a source of training data. This is challenging because real human motion is diverse and adapts to the scene. For example, a person can sit or lie on a sofa in many places and with varying styles. It is necessary to model this diversity when synthesizing virtual humans that realistically perform human-scene interactions. We present a novel data-driven, stochastic motion synthesis method that models different styles of performing a given action with a target object. Our method, called SAMP, for Scene-Aware Motion Prediction, generalizes to target objects of various geometries while enabling the character to navigate in cluttered scenes. To train our method, we collected MoCap data covering various sitting, lying down, walking, and running styles. We demonstrate our method on complex indoor scenes and achieve superior performance compared to existing solutions. Our code and data are available for research at https://samp.is.tue.mpg.de.

Key facts. Cited by 203 papers in the map (data through 7 Oct 2026). Builds on 30 of its 68 references that are in the map. First posted on arXiv 18 Aug 2021.

Builds on

30 of 68 references.

References · 30

  1. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  2. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,600
  3. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,208
  4. 2019Habitat: A Platform for Embodied AI Researchcited by 1,163
  5. 2015Scheduled Sampling for Sequence Prediction with Recurrent Neural Networkscited by 779
  6. 2018Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networkscited by 687
  7. 2020SAPIEN: A SimulAted Part-based Interactive ENvironmentcited by 584
  8. 2018Gibson Env: Real-World Perception for Embodied Agentscited by 477
  9. 2019MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Predictioncited by 417
  10. 2017On human motion prediction using recurrent neural networkscited by 336
  11. 2018SoPhie: An Attentive GAN for Predicting Paths Compliant to Social and Physical Constraintscited by 296
  12. 2013Learning Factored Representations in a Deep Mixture of Expertscited by 275
18 more
  1. 2020Long-term Human Motion Prediction with Scene Contextcited by 191
  2. 2020DLow: Diversifying Latent Flows for Diverse Human Motion Predictioncited by 184
  3. 2021Character Controllers Using Motion VAEscited by 182
  4. 2020Populating 3D Scenes by Learning Human-Scene Interactioncited by 164
  5. 2020iGibson 1.0: a Simulation Environment for Interactive Tasks in Large Realistic Scenescited by 151
  6. 2020Synthesizing Long-Term 3D Human Motion and Interaction in 3D Scenescited by 135
  7. 2019Generating 3D People in Scenes without Peoplecited by 127
  8. 2020We are More than Our Joints: Predicting how 3D Bodies Movecited by 115
  9. 2019Overcoming Limitations of Mixture Density Networks: A Sampling and Fitting Framework for Multimodal Future Predictioncited by 96
  10. 2017Learning to Generate Long-term Future via Hierarchical Predictioncited by 88
  11. 2020Learning to Generate Diverse Dance Motions with Transformercited by 85
  12. 2019Context-aware Human Motion Predictioncited by 83
  13. 2019Diverse Trajectory Forecasting with Determinantal Point Processescited by 80
  14. 2021TRiPOD: Human Trajectory and Pose Dynamics Forecasting in the Wildcited by 52
  15. 2018Binge Watching: Scaling Affordance Learning from Sitcomscited by 41
  16. 2018Combining Recurrent Neural Networks and Adversarial Training for Human Motion Synthesis and Controlcited by 39
  17. 2019Learning to Sit: Synthesizing Human-Chair Interactions via Hierarchical Controlcited by 39
  18. 2020Hierarchical Style-based Networks for Motion Synthesiscited by 23

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 23; 2023: 48; 2024: 55; 2025: 45; 2026: 26.

203 papers.

Sort:

By area:

  1. 2022BEHAVE: Dataset and Method for Tracking Human Object Interactionscited by 246
  2. 2023Diffusion-based Generation, Optimization, and Planning in 3D Scenescited by 237
  3. 2023OmniControl: Control Any Joint at Any Time for Human Motion Generationcited by 233
  4. 2023Motion-X: A Large-scale 3D Expressive Whole-body Human Motion Datasetcited by 220
  5. 2023Object Motion Guided Human Motion Synthesiscited by 206
  6. 2023InterDiff: Generating 3D Human-Object Interactions with Physics-Informed Diffusioncited by 181
  7. 2022TEACH: Temporal Action Composition for 3D Humanscited by 173
  8. 2022HUMANISE: Language-conditioned Human Motion Generation in 3D Scenescited by 156
  9. 2021GLAMR: Global Occlusion-Aware Human Mesh Recovery with Dynamic Camerascited by 151
  10. 2021GOAL: Generating 4D Whole-Body Motion for Hand-Object Graspingcited by 142
  11. 2023Trace and Pace: Controllable Pedestrian Animation via Guided Trajectory Diffusioncited by 137
  12. 2023CG-HOI: Contact-Guided 3D Human-Object Interaction Generationcited by 137
38 more
  1. 2024Scaling Up Dynamic Human-Scene Interaction Modelingcited by 137
  2. 2022COUCH: Towards Controllable Human-Chair Interactionscited by 134
  3. 2022IMos: Intent-Driven Full-Body Motion Synthesis for Human-Object Interactionscited by 121
  4. 2023Synthesizing Diverse Human Motions in 3D Indoor Scenescited by 109
  5. 2023Controllable Human-Object Interaction Synthesiscited by 107
  6. 2023Synthesizing Physical Character-Scene Interactionscited by 102
  7. 2024MaskedMimic: Unified Physics-Based Character Control Through Masked Motion Inpaintingcited by 101
  8. 2023Inter-X: Towards Versatile Human-Human Interaction Analysiscited by 98
  9. 2023Human Motion Generation: A Surveycited by 95
  10. 2023TLControl: Trajectory and Language Control for Human Motion Synthesiscited by 93
  11. 2022Towards Diverse and Natural Scene-aware 3D Human Motion Synthesiscited by 91
  12. 2023Unified Human-Scene Interaction via Prompted Chain-of-Contactscited by 91
  13. 2023HOI-Diff: Text-Driven Synthesis of 3D Human-Object Interactions using Diffusion Modelscited by 89
  14. 2024Multi-Track Timeline Control for Text-Driven 3D Human Motion Generationcited by 88
  15. 2022Compositional Human-Scene Interaction Synthesis with Semantic Controlcited by 82
  16. 2022GIMO: Gaze-Informed Human Motion Prediction in Contextcited by 81
  17. 2023HumanMAC: Masked Motion Completion for Human Motion Predictioncited by 81
  18. 2024Generating Human Interaction Motions in Scenes with Text Controlcited by 80
  19. 2024DartControl: A Diffusion-Based Autoregressive Motion Model for Real-Time Text-Driven Motion Controlcited by 80
  20. 2023CIRCLE: Capture In Rich Contextual Environmentscited by 79
  21. 2024Move as You Say, Interact as You Can: Language-guided Human Motion Generation with Scene Affordancecited by 79
  22. 2023NIFTY: Neural Object Interaction Fields for Guided Human Motion Synthesiscited by 78
  23. 2024Seamless Human Motion Composition with Blended Positional Encodingscited by 77
  24. 2022Full-Body Articulated Human-Object Interactioncited by 76
  25. 2023PhysHOI: Physics-Based Imitation of Dynamic Human-Object Interactioncited by 72
  26. 2025InterMimic: Towards Universal Whole-Body Control for Physics-Based Human-Object Interactionscited by 67
  27. 2024InterDreamer: Zero-Shot Text to 3D Dynamic Human-Object Interactioncited by 63
  28. 2023SINC: Spatial Composition of 3D Human Motions for Simultaneous Action Generationcited by 62
  29. 2024Human-Object Interaction from Human-Level Instructionscited by 62
  30. 2021The Wanderings of Odysseus in 3D Scenescited by 59
  31. 2022MIME: Human-Aware 3D Scene Generationcited by 57
  32. 2024Autonomous Character-Scene Interaction Synthesis from Text Instructioncited by 56
  33. 2024ParaHome: Parameterizing Everyday Home Activities Towards 3D Generative Modeling of Human-Object Interactionscited by 54
  34. 2023ArtiGrasp: Physically Plausible Synthesis of Bi-Manual Dexterous Grasping and Articulationcited by 53
  35. 2023C·ASE: Learning Conditional Adversarial Skill Embeddings for Physics-based Characterscited by 53
  36. 2024ReGenNet: Towards Human Action-Reaction Synthesiscited by 53
  37. 2023Interactive Character Control with Auto-Regressive Motion Diffusion Modelscited by 52
  38. 2023Physically Plausible Full-Body Hand-Object Interaction Synthesiscited by 52

50 most cited of 203 All 203 (JSON)