# Person Image Synthesis via Denoising Diffusion Model

arXiv: https://arxiv.org/abs/2211.12500 · HTML: https://mlsanity.com/paper/2211.12500/

## Key facts

- Cited by 104 papers in the map (data through 14 Oct 2026).
- Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models.
- Builds on 24 of its 30 references that are in the map.
- First posted on arXiv 22 Nov 2022.

## Placed in

- Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/)

## Builds on (most cited first, 24 of 24)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [Pose Guided Person Image Generation](https://mlsanity.com/paper/1705.09368/) (2017)
- [Deformable GANs for Pose-based Human Image Generation](https://mlsanity.com/paper/1801.00055/) (2017)
- [Liquid Warping GAN: A Unified Framework for Human Motion Imitation, Appearance Transfer and Novel View Synthesis](https://mlsanity.com/paper/1909.12224/) (2019)
- [Controllable Person Image Synthesis with Attribute-Decomposed GAN](https://mlsanity.com/paper/2003.12267/) (2020)
- [Progressive Pose Attention Transfer for Person Image Generation](https://mlsanity.com/paper/1904.03349/) (2019)
- [A Variational U-Net for Conditional Appearance and Shape Generation](https://mlsanity.com/paper/1804.04694/) (2018)
- [Cross-domain Correspondence Learning for Exemplar-based Image Translation](https://mlsanity.com/paper/2004.05571/) (2020)
- [Dense Intrinsic Appearance Flow for Human Pose Transfer](https://mlsanity.com/paper/1903.11326/) (2019)
- [Deep Image Spatial Transformation for Person Image Generation](https://mlsanity.com/paper/2003.00696/) (2020)
- [Exploring Dual-task Correlation for Pose Guided Person Image Generation](https://mlsanity.com/paper/2203.02910/) (2022)
- [PISE: Person Image Synthesis and Editing with Decoupled GAN](https://mlsanity.com/paper/2103.04023/) (2021)
- [Neural Texture Extraction and Distribution for Controllable Person Image Synthesis](https://mlsanity.com/paper/2204.06160/) (2022)
- [Cross Attention Based Style Distribution for Controllable Person Image Synthesis](https://mlsanity.com/paper/2208.00712/) (2022)
- [Learning Semantic Person Image Generation by Region-Adaptive Normalization](https://mlsanity.com/paper/2104.06650/) (2021)

## Built on by (most cited first, 30 of 104)

- [Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animation](https://mlsanity.com/paper/2311.17117/) (2023)
- [Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidance](https://mlsanity.com/paper/2403.14781/) (2024)
- [DreamPose: Fashion Image-to-Video Synthesis via Stable Diffusion](https://mlsanity.com/paper/2304.06025/) (2023)
- [LaDI-VTON: Latent Diffusion Textual-Inversion Enhanced Virtual Try-On](https://mlsanity.com/paper/2305.13501/) (2023)
- [Advancing Pose-Guided Image Synthesis with Progressive Conditional Diffusion Models](https://mlsanity.com/paper/2310.06313/) (2023)
- [UniAnimate: Taming Unified Video Diffusion Models for Consistent Human Image Animation](https://mlsanity.com/paper/2406.01188/) (2024)
- [Animate-X: Universal Character Image Animation with Enhanced Motion Representation](https://mlsanity.com/paper/2410.10306/) (2024)
- [Diffusion Action Segmentation](https://mlsanity.com/paper/2303.17959/) (2023)
- [Prompt-Free Diffusion: Taking "Text" out of Text-to-Image Diffusion Models](https://mlsanity.com/paper/2305.16223/) (2023)
- [Reuse and Diffuse: Iterative Denoising for Text-to-Video Generation](https://mlsanity.com/paper/2309.03549/) (2023)
- [Coarse-to-Fine Latent Diffusion for Pose-Guided Person Image Synthesis](https://mlsanity.com/paper/2402.18078/) (2024)
- [Wear-Any-Way: Manipulable Virtual Try-on via Sparse Correspondence Alignment](https://mlsanity.com/paper/2403.12965/) (2024)
- [RealisDance: Equip controllable character animation with realistic hands](https://mlsanity.com/paper/2409.06202/) (2024)
- [Enabling Synergistic Full-Body Control in Prompt-Based Co-Speech Motion Generation](https://mlsanity.com/paper/2410.00464/) (2024)
- [Enhance Image Classification via Inter-Class Image Mixup with Diffusion Model](https://mlsanity.com/paper/2403.19600/) (2024)
- [How I Warped Your Noise: a Temporally-Correlated Noise Prior for Diffusion Models](https://mlsanity.com/paper/2504.03072/) (2025)
- [VividPose: Advancing Stable Video Diffusion for Realistic Human Image Animation](https://mlsanity.com/paper/2405.18156/) (2024)
- [Learning Flow Fields in Attention for Controllable Person Image Generation](https://mlsanity.com/paper/2412.08486/) (2024)
- [SmartControl: Enhancing ControlNet for Handling Rough Visual Conditions](https://mlsanity.com/paper/2404.06451/) (2024)
- [MagicMan: Generative Novel View Synthesis of Humans with 3D-Aware Diffusion and Iterative Refinement](https://mlsanity.com/paper/2408.14211/) (2024)
- [GaussianVTON: 3D Human Virtual Try-ON via Multi-Stage Gaussian Splatting Editing with Image Prompting](https://mlsanity.com/paper/2405.07472/) (2024)
- [TALK-Act: Enhance Textural-Awareness for 2D Speaking Avatar Reenactment with Diffusion Model](https://mlsanity.com/paper/2410.10696/) (2024)
- [Re-HOLD: Video Hand Object Interaction Reenactment via adaptive Layout-instructed Diffusion Model](https://mlsanity.com/paper/2503.16942/) (2025)
- [From Poses to Identity: Training-Free Person Re-Identification via Feature Centralization](https://mlsanity.com/paper/2503.00938/) (2025)
- [PICTURE: PhotorealistIC virtual Try-on from UnconstRained dEsigns](https://mlsanity.com/paper/2312.04534/) (2023)
- [Giving a Hand to Diffusion Models: a Two-Stage Approach to Improving Conditional Human Image Generation](https://mlsanity.com/paper/2403.10731/) (2024)
- [UPGPT: Universal Diffusion Model for Person Image Generation, Editing and Pose Transfer](https://mlsanity.com/paper/2304.08870/) (2023)
- [UniHuman: A Unified Model for Editing Human Images in the Wild](https://mlsanity.com/paper/2312.14985/) (2023)
- [Cross-view Masked Diffusion Transformers for Person Image Synthesis](https://mlsanity.com/paper/2402.01516/) (2024)
- [Stable-Pose: Leveraging Transformers for Pose-Guided Text-to-Image Generation](https://mlsanity.com/paper/2406.02485/) (2024)

## Abstract (arXiv)

The pose-guided person image generation task requires synthesizing photorealistic images of humans in arbitrary poses. The existing approaches use generative adversarial networks that do not necessarily maintain realistic textures or need dense correspondences that struggle to handle complex deformations and severe occlusions. In this work, we show how denoising diffusion models can be applied for high-fidelity person image synthesis with strong sample diversity and enhanced mode coverage of the learnt data distribution. Our proposed Person Image Diffusion Model (PIDM) disintegrates the complex transfer problem into a series of simpler forward-backward denoising steps. This helps in learning plausible source-to-target transformation trajectories that result in faithful textures and undistorted appearance details. We introduce a 'texture diffusion module' based on cross-attention to accurately model the correspondences between appearance and pose information available in source and target images. Further, we propose 'disentangled classifier-free guidance' to ensure close resemblance between the conditional inputs and the synthesized output in terms of both pose and appearance information. Our extensive results on two large-scale benchmarks and a user study demonstrate the photorealism of our proposed approach under challenging scenarios. We also show how our generated images can help in downstream tasks. Our code and models will be publicly released.