# Synthesizing Long-Term 3D Human Motion and Interaction in 3D Scenes

arXiv: https://arxiv.org/abs/2012.05522 · HTML: https://mlsanity.com/paper/2012.05522/

## Key facts

- Cited by 135 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Dec 2020.

## Built on by (most cited first, 30 of 135)

- [OpenScene: 3D Scene Understanding with Open Vocabularies](https://mlsanity.com/paper/2211.15654/) (2022)
- [Diffusion-based Generation, Optimization, and Planning in 3D Scenes](https://mlsanity.com/paper/2301.06015/) (2023)
- [Guided Motion Diffusion for Controllable Human Motion Synthesis](https://mlsanity.com/paper/2305.12577/) (2023)
- [Object Motion Guided Human Motion Synthesis](https://mlsanity.com/paper/2309.16237/) (2023)
- [Stochastic Scene-Aware Motion Prediction](https://mlsanity.com/paper/2108.08284/) (2021)
- [InterDiff: Generating 3D Human-Object Interactions with Physics-Informed Diffusion](https://mlsanity.com/paper/2308.16905/) (2023)
- [Hand-Object Contact Consistency Reasoning for Human Grasps Generation](https://mlsanity.com/paper/2104.03304/) (2021)
- [HUMANISE: Language-conditioned Human Motion Generation in 3D Scenes](https://mlsanity.com/paper/2210.09729/) (2022)
- [GOAL: Generating 4D Whole-Body Motion for Hand-Object Grasping](https://mlsanity.com/paper/2112.11454/) (2021)
- [CG-HOI: Contact-Guided 3D Human-Object Interaction Generation](https://mlsanity.com/paper/2311.16097/) (2023)
- [Scaling Up Dynamic Human-Scene Interaction Modeling](https://mlsanity.com/paper/2403.08629/) (2024)
- [COUCH: Towards Controllable Human-Chair Interactions](https://mlsanity.com/paper/2205.00541/) (2022)
- [PhyScene: Physically Interactable 3D Scene Synthesis for Embodied AI](https://mlsanity.com/paper/2404.09465/) (2024)
- [Joint Hand Motion and Interaction Hotspots Prediction from Egocentric Videos](https://mlsanity.com/paper/2204.01696/) (2022)
- [Synthesizing Diverse Human Motions in 3D Indoor Scenes](https://mlsanity.com/paper/2305.12411/) (2023)
- [Controllable Human-Object Interaction Synthesis](https://mlsanity.com/paper/2312.03913/) (2023)
- [Synthesizing Physical Character-Scene Interactions](https://mlsanity.com/paper/2302.00883/) (2023)
- [Optimizing Diffusion Noise Can Serve As Universal Motion Priors](https://mlsanity.com/paper/2312.11994/) (2023)
- [Human Motion Generation: A Survey](https://mlsanity.com/paper/2307.10894/) (2023)
- [Ego-Body Pose Estimation via Ego-Head Pose Estimation](https://mlsanity.com/paper/2212.04636/) (2022)
- [SAGA: Stochastic Whole-Body Grasping with Contact](https://mlsanity.com/paper/2112.10103/) (2021)
- [Towards Diverse and Natural Scene-aware 3D Human Motion Synthesis](https://mlsanity.com/paper/2205.13001/) (2022)
- [Multi-Track Timeline Control for Text-Driven 3D Human Motion Generation](https://mlsanity.com/paper/2401.08559/) (2024)
- [Compositional Human-Scene Interaction Synthesis with Semantic Control](https://mlsanity.com/paper/2207.12824/) (2022)
- [GIMO: Gaze-Informed Human Motion Prediction in Context](https://mlsanity.com/paper/2204.09443/) (2022)
- [Seamless Human Motion Composition with Blended Positional Encodings](https://mlsanity.com/paper/2402.15509/) (2024)
- [Move as You Say, Interact as You Can: Language-guided Human Motion Generation with Scene Affordance](https://mlsanity.com/paper/2403.18036/) (2024)
- [Generating Human Interaction Motions in Scenes with Text Control](https://mlsanity.com/paper/2404.10685/) (2024)
- [Full-Body Articulated Human-Object Interaction](https://mlsanity.com/paper/2212.10621/) (2022)
- [NIFTY: Neural Object Interaction Fields for Guided Human Motion Synthesis](https://mlsanity.com/paper/2307.07511/) (2023)

## Abstract (arXiv)

Synthesizing 3D human motion plays an important role in many graphics applications as well as understanding human activity. While many efforts have been made on generating realistic and natural human motion, most approaches neglect the importance of modeling human-scene interactions and affordance. On the other hand, affordance reasoning (e.g., standing on the floor or sitting on the chair) has mainly been studied with static human pose and gestures, and it has rarely been addressed with human motion. In this paper, we propose to bridge human motion synthesis and scene affordance reasoning. We present a hierarchical generative framework to synthesize long-term 3D human motion conditioning on the 3D scene structure. Building on this framework, we further enforce multiple geometry constraints between the human mesh and scene point clouds via optimization to improve realistic synthesis. Our experiments show significant improvements over previous approaches on generating natural and physically plausible human motion in a scene.