# Placing Human Animations into 3D Scenes by Learning Interaction- and Geometry-Driven Keyframes

arXiv: https://arxiv.org/abs/2209.06314 · HTML: https://mlsanity.com/paper/2209.06314/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 67 references that are in the map.
- First posted on arXiv 13 Sep 2022.

## Builds on (most cited first, 30 of 30)

- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [Efficient Geometry-aware 3D Generative Adversarial Networks](https://mlsanity.com/paper/2112.07945/) (2021)
- [GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields](https://mlsanity.com/paper/2011.12100/) (2020)
- [Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Bounds](https://mlsanity.com/paper/1906.03671/) (2019)
- [StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis](https://mlsanity.com/paper/2110.08985/) (2021)
- [Animatable Neural Radiance Fields for Modeling Dynamic Human Bodies](https://mlsanity.com/paper/2105.02872/) (2021)
- [HuMoR: 3D Human Motion Model for Robust Pose Estimation](https://mlsanity.com/paper/2105.04668/) (2021)
- [Resolving 3D Human Pose Ambiguities with 3D Scene Constraints](https://mlsanity.com/paper/1908.06963/) (2019)
- [Critic Regularized Regression](https://mlsanity.com/paper/2006.15134/) (2020)
- [Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodies](https://mlsanity.com/paper/1801.01615/) (2018)
- [A-NeRF: Articulated Neural Radiance Fields for Learning Human Shape, Appearance, and Pose](https://mlsanity.com/paper/2102.06199/) (2021)
- [Robust Motion In-betweening](https://mlsanity.com/paper/2102.04942/) (2021)
- [BANMo: Building Animatable 3D Neural Models from Many Casual Videos](https://mlsanity.com/paper/2112.12761/) (2021)
- [Neural Articulated Radiance Field](https://mlsanity.com/paper/2104.03110/) (2021)
- [STAR: Sparse Trained Articulated Human Body Regressor](https://mlsanity.com/paper/2008.08535/) (2020)
- [Character Controllers Using Motion VAEs](https://mlsanity.com/paper/2103.14274/) (2021)
- [Populating 3D Scenes by Learning Human-Scene Interaction](https://mlsanity.com/paper/2012.11581/) (2020)
- [Neural Human Performer: Learning Generalizable Radiance Fields for Human Performance Rendering](https://mlsanity.com/paper/2109.07448/) (2021)
- [Synthesizing Long-Term 3D Human Motion and Interaction in 3D Scenes](https://mlsanity.com/paper/2012.05522/) (2020)
- [Generating 3D People in Scenes without People](https://mlsanity.com/paper/1912.02923/) (2019)
- [HumanNeRF: Efficiently Generated Human Radiance Field from Sparse Inputs](https://mlsanity.com/paper/2112.02789/) (2021)
- [Towards Diverse and Natural Scene-aware 3D Human Motion Synthesis](https://mlsanity.com/paper/2205.13001/) (2022)
- [Active Adversarial Domain Adaptation](https://mlsanity.com/paper/1904.07848/) (2019)
- [Putting Humans in a Scene: Learning Affordance in 3D Indoor Environments](https://mlsanity.com/paper/1903.05690/) (2019)
- [Active Domain Adaptation via Clustering Uncertainty-weighted Embeddings](https://mlsanity.com/paper/2010.08666/) (2020)
- [Unsupervised Learning of Efficient Geometry-Aware Neural Articulated Representations](https://mlsanity.com/paper/2204.08839/) (2022)
- [Hierarchical Style-based Networks for Motion Synthesis](https://mlsanity.com/paper/2008.10162/) (2020)
- [Where and Who? Automatic Semantic-Aware Person Composition](https://mlsanity.com/paper/1706.01021/) (2017)
- [Pose to Seat: Automated Design of Body-Supporting Surfaces](https://mlsanity.com/paper/2003.10435/) (2020)

## Built on by (most cited first, 8 of 8)

- [CG-HOI: Contact-Guided 3D Human-Object Interaction Generation](https://mlsanity.com/paper/2311.16097/) (2023)
- [A Survey on Human Interaction Motion Generation](https://mlsanity.com/paper/2503.12763/) (2025)
- [Sitcom-Crafter: A Plot-Driven Human Motion Generation System in 3D Scenes](https://mlsanity.com/paper/2410.10790/) (2024)
- [Diffusion Implicit Policy for Unpaired Scene-aware Motion Synthesis](https://mlsanity.com/paper/2412.02261/) (2024)
- [PACE: Data-Driven Virtual Agent Interaction in Dense and Cluttered Environments](https://mlsanity.com/paper/2303.14255/) (2023)
- [DEMOS: Dynamic Environment Motion Synthesis in 3D Scenes via Local Spherical-BEV Perception](https://mlsanity.com/paper/2403.01740/) (2024)
- [ARMS: Anchor-Relational Motion Streaming for Seamless Solo-Social Motion Transitions](https://mlsanity.com/paper/2607.05733/) (2026)
- [Efficient Human-Contact Representation for Human-Scene Interaction](https://mlsanity.com/paper/2608.09388/) (2026)

## Abstract (arXiv)

We present a novel method for placing a 3D human animation into a 3D scene while maintaining any human-scene interactions in the animation. We use the notion of computing the most important meshes in the animation for the interaction with the scene, which we call "keyframes." These keyframes allow us to better optimize the placement of the animation into the scene such that interactions in the animations (standing, laying, sitting, etc.) match the affordances of the scene (e.g., standing on the floor or laying in a bed). We compare our method, which we call PAAK, with prior approaches, including POSA, PROX ground truth, and a motion synthesis method, and highlight the benefits of our method with a perceptual study. Human raters preferred our PAAK method over the PROX ground truth data 64.6% of the time. Additionally, in direct comparisons, the raters preferred PAAK over competing methods including 61.5% compared to POSA.