mlsanityData to 7 Oct 2026

UPGPT: Universal Diffusion Model for Person Image Generation, Editing and Pose Transfer

arXiv 2304.08870 ↗First posted 18 Apr 2023cs.CV cs.AIcited by 12

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

Text-to-image models (T2I) such as StableDiffusion have been used to generate high quality images of people. However, due to the random nature of the generation process, the person has a different appearance e.g. pose, face, and clothing, despite using the same text prompt. The appearance inconsistency makes T2I unsuitable for pose transfer. We address this by proposing a multimodal diffusion model that accepts text, pose, and visual prompting. Our model is the first unified method to perform all person image tasks - generation, pose transfer, and mask-less edit. We also pioneer using small dimensional 3D body model parameters directly to demonstrate new capability - simultaneous pose and camera view interpolation while maintaining the person's appearance.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 45 of its 56 references that are in the map. First posted on arXiv 18 Apr 2023.

Builds on

45 of 56 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Person Image Synthesis via Denoising Diffusion Model22 Nov

Other references · 44

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  6. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  7. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  8. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  9. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  10. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  11. 2022Classifier-Free Diffusion Guidancecited by 5,717
  12. 2022Generative Adversarial Networkscited by 5,603
32 more
  1. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  2. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  3. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  4. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  5. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  6. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  7. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,678
  8. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  9. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,295
  10. 2021Blended Diffusion for Text-driven Editing of Natural Imagescited by 924
  11. 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
  12. 2016StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 575
  13. 2018DensePose: Dense Human Pose Estimation In The Wildcited by 573
  14. 2022Blended Latent Diffusioncited by 430
  15. 2019DM-GAN: Dynamic Memory Generative Adversarial Networks for Text-to-Image Synthesiscited by 308
  16. 2022VQGAN-CLIP: Open Domain Image Generation and Editing with Natural Language Guidancecited by 285
  17. 2021Cross-Modal Contrastive Learning for Text-to-Image Generationcited by 259
  18. 2017Pose Guided Person Image Generationcited by 251
  19. 2017Deformable GANs for Pose-based Human Image Generationcited by 166
  20. 2020DF-GAN: A Simple and Effective Baseline for Text-to-Image Synthesiscited by 161
  21. 2020Perceiving 3D Human-Object Spatial Arrangements from a Single Image in the Wildcited by 152
  22. 2020Controllable Person Image Synthesis with Attribute-Decomposed GANcited by 150
  23. 2019Progressive Pose Attention Transfer for Person Image Generationcited by 138
  24. 2020Deep Image Spatial Transformation for Person Image Generationcited by 100
  25. 2022Exploring Dual-task Correlation for Pose Guided Person Image Generationcited by 68
  26. 2021PISE: Person Image Synthesis and Editing with Decoupled GANcited by 62
  27. 2022Neural Texture Extraction and Distribution for Controllable Person Image Synthesiscited by 55
  28. 2022Cross Attention Based Style Distribution for Controllable Person Image Synthesiscited by 45
  29. 2021Learning Semantic Person Image Generation by Region-Adaptive Normalizationcited by 44
  30. 2020Towards Fine-grained Human Pose Transfer with Detail Replenishing Networkcited by 21
  31. 2022KPE: Keypoint Pose Encoding for Transformer-based Image Generationcited by 12
  32. 2022HumanDiffusion: a Coarse-to-Fine Alignment Diffusion Framework for Controllable Text-Driven Person Image Generationcited by 11

Built on by

Cited by year

Citing papers per year: 2023: 3; 2024: 5; 2025: 4.

12 papers.

  1. 2023ProRes: Exploring Degradation-aware Visual Prompt for Universal Image Restorationcited by 77
  2. 2024Generative Artificial Intelligence: A Systematic Review and Applicationscited by 37
  3. 2024GaussianVTON: 3D Human Virtual Try-ON via Multi-Stage Gaussian Splatting Editing with Image Promptingcited by 22
  4. 2024Boosting Camera Motion Control for Video Diffusion Transformerscited by 20
  5. 2024TCAN: Animating Human Images with Temporally Consistent Pose Guidance using Diffusion Modelscited by 15
  6. 2023UniHuman: A Unified Model for Editing Human Images in the Wildcited by 12
  7. 2023ViscoNet: Bridging and Harmonizing Visual and Textual Conditioning for ControlNetcited by 7
  8. 2025Multi-focal Conditioned Latent Diffusion for Person Image Synthesiscited by 6
  9. 2025Rethink Sparse Signals for Pose-guided Text-to-image Generationcited by 5
  10. 2024Bridging the Intent Gap: Knowledge-Enhanced Visual Generationcited by 1
  11. 2025Computationally Efficient Diffusion Models in Medical Imaging: A Comprehensive Reviewcited by 1
  12. 2025CraftGraffiti: Exploring Human Identity with Custom Graffiti Art via Facial-Preserving Diffusion Modelscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2024Coarse-to-Fine Latent Diffusion for Pose-Guided Person Image Synthesis
  2. 2023Advancing Pose-Guided Image Synthesis with Progressive Conditional Diffusion Models
  3. 2024From Text to Pose to Image: Improving Diffusion Model Control and Quality
  4. 2026One-Shot Novel View and Pose Human Image Synthesis via 3D Prior Guided Diffusion Model
  5. 2024Human Multi-View Synthesis from a Single-View Model:Transferred Body and Face Representations
  6. 2024Synthesizing Moving People with 3D Control
  7. 2023High-fidelity Person-centric Subject-to-Image Synthesis
  8. 2023Single-Image 3D Human Digitization with Shape-Guided Diffusion