mlsanityData to 14 Oct 2026

Putting People in Their Place: Affordance-Aware Human Insertion into Scenes

arXiv 2304.14406 ↗First posted 27 Apr 2023cs.CV cs.AI cs.GRcited by 53

Not on the map. Why?

Abstract (arXiv)

We study the problem of inferring scene affordances by presenting a method for realistically inserting people into scenes. Given a scene image with a marked region and an image of a person, we insert the person into the scene while respecting the scene affordances. Our model can infer the set of realistic poses given the scene context, re-pose the reference person, and harmonize the composition. We set up the task in a self-supervised fashion by learning to re-pose humans in video clips. We train a large-scale diffusion model on a dataset of 2.4M video clips that produces diverse plausible poses while respecting the scene context. Given the learned human-scene composition, our model can also hallucinate realistic people and scenes when prompted without conditioning and also enables interactive editing. A quantitative evaluation shows that our method synthesizes more realistic human appearance and more natural human-scene interactions than prior work.

Key facts. Cited by 53 papers in the map (data through 14 Oct 2026). Builds on 54 of its 78 references that are in the map. First posted on arXiv 27 Apr 2023.

Builds on

54 of 78 references.

References · 54

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2020Denoising Diffusion Implicit Modelscited by 9,781
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  6. 2017Mask R-CNNcited by 7,277
  7. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  8. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  9. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  10. 2022Classifier-Free Diffusion Guidancecited by 5,847
  11. 2014Going Deeper with Convolutionscited by 4,241
  12. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
38 more
  1. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
  2. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  3. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  4. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,778
  5. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
  6. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,430
  7. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,428
  8. 2021Palette: Image-to-Image Diffusion Modelscited by 1,300
  9. 2020Training Generative Adversarial Networks with Limited Datacited by 1,186
  10. 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,162
  11. 2019CNN-generated images are surprisingly easy to spot... for nowcited by 847
  12. 2020First Order Motion Model for Image Animationcited by 704
  13. 2018Free-Form Image Inpainting with Gated Convolutioncited by 594
  14. 2018Generative Image Inpainting with Contextual Attentioncited by 580
  15. 2017MoCoGAN: Decomposing Motion and Content for Video Generationcited by 524
  16. 2018Image Inpainting for Irregular Holes Using Partial Convolutionscited by 505
  17. 2018Video-to-Video Synthesiscited by 390
  18. 2018Everybody Dance Nowcited by 315
  19. 2017Pose Guided Person Image Generationcited by 251
  20. 2022Visual Prompting via Image Inpaintingcited by 250
  21. 2021Large Scale Image Completion via Co-Modulated Generative Adversarial Networkscited by 220
  22. 2017Detecting and Recognizing Human-Object Interactionscited by 199
  23. 2020Long-term Human Motion Prediction with Scene Contextcited by 192
  24. 2017Deformable GANs for Pose-based Human Image Generationcited by 166
  25. 2020Populating 3D Scenes by Learning Human-Scene Interactioncited by 164
  26. 2020Reconstructing Hand-Object Interactions in the Wildcited by 144
  27. 2020Synthesizing Long-Term 3D Human Motion and Interaction in 3D Scenescited by 135
  28. 2019Dense Intrinsic Appearance Flow for Human Pose Transfercited by 104
  29. 2018Synthesizing Images of Humans in Unseen Posescited by 97
  30. 2016High-Resolution Image Inpainting using Multi-Scale Neural Patch Synthesiscited by 90
  31. 2017Learning to Generate Long-term Future via Hierarchical Predictioncited by 88
  32. 2018Dense Pose Transfercited by 83
  33. 2021Diverse Image Inpainting with Bidirectional and Autoregressive Transformerscited by 80
  34. 2017Learning to Act Properly: Predicting and Explaining Affordances from Imagescited by 74
  35. 2019Putting Humans in a Scene: Learning Affordance in 3D Indoor Environmentscited by 72
  36. 2020Pose-Guided Human Animation from a Single Image in the Wildcited by 53
  37. 2022CM-GAN: Image Inpainting with Cascaded Modulation GAN and Object-Aware Trainingcited by 49
  38. 2021Pose with Style: Detail-Preserving Pose-Guided Image Synthesis with Conditional StyleGANcited by 45

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 8; 2024: 20; 2025: 17; 2026: 8.

53 papers.

Sort:

By area:

  1. 2023A Survey of Hallucination in Large Foundation Modelscited by 368
  2. 2023VideoBooth: Diffusion-based Video Generation with Image Promptscited by 126
  3. 2024A Comprehensive Survey of Hallucination in Large Language, Image, Video and Audio Foundation Modelscited by 100
  4. 2024Move as You Say, Interact as You Can: Language-guided Human Motion Generation with Scene Affordancecited by 80
  5. 2023Diffusion-Guided Reconstruction of Everyday Hand-Object Interaction Clipscited by 74
  6. 2024ObjectDrop: Bootstrapping Counterfactuals for Photorealistic Object Removal and Insertioncited by 71
  7. 2025Animate Anyone 2: High-Fidelity Character Image Animation with Environment Affordancecited by 61
  8. 2025Insert Anything: Image Insertion via In-Context Editing in DiTcited by 61
  9. 2024AffordanceLLM: Grounding Affordance from Vision Language Modelscited by 55
  10. 2024MoA: Mixture-of-Attention for Subject-Context Disentanglement in Personalized Image Generationcited by 51
  11. 2023ControlCom: Controllable Image Composition using Diffusion Modelcited by 43
  12. 2023GenHowTo: Learning to Generate Actions and State Transformations from Instructional Videoscited by 40
38 more
  1. 2024Beyond the Contact: Discovering Comprehensive Affordance for 3D Objects from Pre-trained 2D Diffusion Modelscited by 32
  2. 2023Paste, Inpaint and Harmonize via Denoising: Subject-Driven Image Editing with Pre-Trained Diffusion Modelcited by 21
  3. 2024HOI-Swap: Swapping Objects in Videos with Hand-Object Interaction Awarenesscited by 21
  4. 2024Thinking Outside the BBox: Unconstrained Generative Object Compositingcited by 19
  5. 2023DreamCom: Finetuning Text-guided Inpainting Model for Image Compositioncited by 17
  6. 2024LLM-enhanced Scene Graph Learning for Household Rearrangementcited by 12
  7. 2024InterDyn: Controllable Interactive Dynamics with Video Diffusion Modelscited by 12
  8. 2024Text2Place: Affordance-aware Text Guided Human Placementcited by 11
  9. 2025Mask2IV: Interaction-Centric Video Generation via Mask Trajectoriescited by 10
  10. 2024TALE: Training-free Cross-domain Image Composition via Adaptive Latent Manipulation and Energy-guided Optimizationcited by 9
  11. 2025Multitwine: Multi-Object Compositing with Text and Layout Controlcited by 9
  12. 2024ActAnywhere: Subject-Aware Video Background Generationcited by 8
  13. 2024GraspDiffusion: Synthesizing Realistic Whole-body Hand-Object Interactioncited by 8
  14. 2024Move-in-2D: 2D-Conditioned Human Motion Generationcited by 8
  15. 2024Affordance-Aware Object Insertion via Mask-Aware Dual Diffusioncited by 6
  16. 2025GenHSI: Controllable Generation of Human-Scene Interaction Videoscited by 6
  17. 2024Synthesizing Moving People with 3D Controlcited by 5
  18. 2023Synthesizing Environment-Specific People in Photographscited by 4
  19. 2024LightHouse: A Survey of AGI Hallucinationcited by 4
  20. 2025MObI: Multimodal Object Inpainting Using Diffusion Modelscited by 4
  21. 2025DAViD: Modeling Dynamic Affordance of 3D Objects Using Pre-trained Video Diffusion Modelscited by 4
  22. 2025MonoPlace3D: Learning 3D-Aware Object Placement for 3D Monocular Detectioncited by 4
  23. 2025FairyGen: Storied Cartoon Video from a Single Child-Drawn Charactercited by 4
  24. 2026MotionWeaver: Holistic 4D-Anchored Framework for Multi-Humanoid Image Animationcited by 3
  25. 2024GroupDiff: Diffusion-based Group Portrait Editingcited by 2
  26. 2024Potential Field as Scene Affordance for Behavior Change-Based Visual Risk Object Identificationcited by 2
  27. 2025MoCA-Video: Motion-Aware Concept Alignment for Consistent Video Editingcited by 2
  28. 2025MonSTeR: a Unified Model for Motion, Scene, Text Retrievalcited by 2
  29. 2025Teleportraits: Training-Free People Insertion into Any Scenecited by 2
  30. 2026A2-Edit: Precise Reference-Guided Image Editing of Arbitrary Objects and Ambiguous Maskscited by 2
  31. 2025Populate-A-Scene: Affordance-Aware Human Video Generationcited by 1
  32. 2026InHabit: Leveraging Image Foundation Models for Scalable 3D Human Placementcited by 1
  33. 2026InsHuman: Towards Natural and Identity-Preserving Human Insertioncited by 1
  34. 2025Towards Affordance-Aware Articulation Synthesis for Rigged Objectscited by 0
  35. 2025SwapAnyone: Consistent and Realistic Video Synthesis for Swapping Any Person into Any Videocited by 0
  36. 2025Person-In-Situ: Scene-Consistent Human Image Insertion with Occlusion-Aware Pose Controlcited by 0
  37. 2025Reference-Guided Diffusion Inpainting For Multimodal Counterfactual Generationcited by 0
  38. 2026VSDiffusion: Taming Ill-Posed Shadow Generation via Visibility-Constrained Diffusioncited by 0

50 most cited of 53 All 53 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Scene-aware Human Pose Generation using Transformer
  2. 2024WorldAfford: Affordance Grounding based on Natural Language Instructions
  3. 2026Affordance2Action: Task-Conditioned Scene-level Affordance Grounding for Real-Time Manipulation ↗
  4. 2024Robo-ABC: Affordance Generalization Beyond Categories via Semantic Correspondence for Robot Manipulation
  5. 2023Affordance Diffusion: Synthesizing Hand-Object Interactions
  6. 2026VAGNet: Grounding 3D Affordance from Human-Object Interactions in Videos
  7. 2025InteractAnything: Zero-shot Human Object Interaction Synthesis via LLM Feedback and Object Affordance Parsing
  8. 2024INTRA: Interaction Relationship-aware Weakly Supervised Affordance Grounding