# Pose Guided Human Image Synthesis with Partially Decoupled GAN

arXiv: https://arxiv.org/abs/2210.03627 · HTML: https://mlsanity.com/paper/2210.03627/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 34 references that are in the map.
- First posted on arXiv 7 Oct 2022.

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [Self-Attention Generative Adversarial Networks](https://mlsanity.com/paper/1805.08318/) (2018)
- [Pose Guided Person Image Generation](https://mlsanity.com/paper/1705.09368/) (2017)
- [Deformable GANs for Pose-based Human Image Generation](https://mlsanity.com/paper/1801.00055/) (2017)
- [Controllable Person Image Synthesis with Attribute-Decomposed GAN](https://mlsanity.com/paper/2003.12267/) (2020)
- [Progressive Pose Attention Transfer for Person Image Generation](https://mlsanity.com/paper/1904.03349/) (2019)
- [A Variational U-Net for Conditional Appearance and Shape Generation](https://mlsanity.com/paper/1804.04694/) (2018)
- [Generative Adversarial Transformers](https://mlsanity.com/paper/2103.01209/) (2021)
- [Dense Intrinsic Appearance Flow for Human Pose Transfer](https://mlsanity.com/paper/1903.11326/) (2019)
- [Deep Image Spatial Transformation for Person Image Generation](https://mlsanity.com/paper/2003.00696/) (2020)
- [XingGAN for Person Image Generation](https://mlsanity.com/paper/2007.09278/) (2020)
- [Exploring Dual-task Correlation for Pose Guided Person Image Generation](https://mlsanity.com/paper/2203.02910/) (2022)
- [PISE: Person Image Synthesis and Editing with Decoupled GAN](https://mlsanity.com/paper/2103.04023/) (2021)
- [Neural Texture Extraction and Distribution for Controllable Person Image Synthesis](https://mlsanity.com/paper/2204.06160/) (2022)
- [Dressing in Order: Recurrent Person Image Generation for Pose Transfer, Virtual Try-on and Outfit Editing](https://mlsanity.com/paper/2104.07021/) (2021)
- [Learning Semantic Person Image Generation by Region-Adaptive Normalization](https://mlsanity.com/paper/2104.06650/) (2021)
- [Cloth Interactive Transformer for Virtual Try-On](https://mlsanity.com/paper/2104.05519/) (2021)
- [nnSpeech: Speaker-Guided Conditional Variational Autoencoder for Zero-shot Multi-speaker Text-to-Speech](https://mlsanity.com/paper/2202.10712/) (2022)
- [TGAVC: Improving Autoencoder Voice Conversion with Text-Guided and Adversarial Training](https://mlsanity.com/paper/2208.04035/) (2022)
- [Improving Human Image Synthesis with Residual Fast Fourier Transformation and Wasserstein Distance](https://mlsanity.com/paper/2205.12022/) (2022)
- [Augmentation-induced Consistency Regularization for Classification](https://mlsanity.com/paper/2205.12461/) (2022)

## Built on by (most cited first, 3 of 3)

- [Cross-view Masked Diffusion Transformers for Person Image Synthesis](https://mlsanity.com/paper/2402.01516/) (2024)
- [Transformer-based Generative Adversarial Networks in Computer Vision: A Comprehensive Survey](https://mlsanity.com/paper/2302.08641/) (2023)
- [AOSR-Net: All-in-One Sandstorm Removal Network](https://mlsanity.com/paper/2309.08838/) (2023)

## Abstract (arXiv)

Pose Guided Human Image Synthesis (PGHIS) is a challenging task of transforming a human image from the reference pose to a target pose while preserving its style. Most existing methods encode the texture of the whole reference human image into a latent space, and then utilize a decoder to synthesize the image texture of the target pose. However, it is difficult to recover the detailed texture of the whole human image. To alleviate this problem, we propose a method by decoupling the human body into several parts (eg, hair, face, hands, feet, etc) and then using each of these parts to guide the synthesis of a realistic image of the person, which preserves the detailed information of the generated images. In addition, we design a multi-head attention-based module for PGHIS. Because most convolutional neural network-based methods have difficulty in modeling long-range dependency due to the convolutional operation, the long-range modeling capability of attention mechanism is more suitable than convolutional neural networks for pose transfer task, especially for sharp pose deformation. Extensive experiments on Market-1501 and DeepFashion datasets reveal that our method almost outperforms other existing state-of-the-art methods in terms of both qualitative and quantitative metrics.