# Exploring Dual-task Correlation for Pose Guided Person Image Generation

arXiv: https://arxiv.org/abs/2203.02910 · HTML: https://mlsanity.com/paper/2203.02910/

## Key facts

- Cited by 68 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 46 references that are in the map.
- First posted on arXiv 6 Mar 2022.

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [An Overview of Multi-Task Learning in Deep Neural Networks](https://mlsanity.com/paper/1706.05098/) (2017)
- [Pose Guided Person Image Generation](https://mlsanity.com/paper/1705.09368/) (2017)
- [Improved Texture Networks: Maximizing Quality and Diversity in Feed-forward Stylization and Texture Synthesis](https://mlsanity.com/paper/1701.02096/) (2017)
- [Deformable GANs for Pose-based Human Image Generation](https://mlsanity.com/paper/1801.00055/) (2017)
- [FastReID: A Pytorch Toolbox for General Instance Re-identification](https://mlsanity.com/paper/2006.02631/) (2020)
- [Dual Learning for Machine Translation](https://mlsanity.com/paper/1611.00179/) (2016)
- [Progressive Pose Attention Transfer for Person Image Generation](https://mlsanity.com/paper/1904.03349/) (2019)
- [A Variational U-Net for Conditional Appearance and Shape Generation](https://mlsanity.com/paper/1804.04694/) (2018)
- [Generative Adversarial Transformers](https://mlsanity.com/paper/2103.01209/) (2021)
- [Dense Intrinsic Appearance Flow for Human Pose Transfer](https://mlsanity.com/paper/1903.11326/) (2019)
- [Deep Image Spatial Transformation for Person Image Generation](https://mlsanity.com/paper/2003.00696/) (2020)
- [XingGAN for Person Image Generation](https://mlsanity.com/paper/2007.09278/) (2020)
- [PISE: Person Image Synthesis and Editing with Decoupled GAN](https://mlsanity.com/paper/2103.04023/) (2021)
- [Spatial-Temporal Person Re-identification](https://mlsanity.com/paper/1812.03282/) (2018)
- [Learning Semantic Person Image Generation by Region-Adaptive Normalization](https://mlsanity.com/paper/2104.06650/) (2021)
- [Learning View-Specific Deep Networks for Person Re-Identification](https://mlsanity.com/paper/1803.11333/) (2018)

## Built on by (most cited first, 30 of 68)

- [Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animation](https://mlsanity.com/paper/2311.17117/) (2023)
- [Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidance](https://mlsanity.com/paper/2403.14781/) (2024)
- [HumanSD: A Native Skeleton-Guided Diffusion Model for Human Image Generation](https://mlsanity.com/paper/2304.04269/) (2023)
- [Person Image Synthesis via Denoising Diffusion Model](https://mlsanity.com/paper/2211.12500/) (2022)
- [Advancing Pose-Guided Image Synthesis with Progressive Conditional Diffusion Models](https://mlsanity.com/paper/2310.06313/) (2023)
- [UniAnimate: Taming Unified Video Diffusion Models for Consistent Human Image Animation](https://mlsanity.com/paper/2406.01188/) (2024)
- [Animate-X: Universal Character Image Animation with Enhanced Motion Representation](https://mlsanity.com/paper/2410.10306/) (2024)
- [HyperHuman: Hyper-Realistic Human Generation with Latent Structural Diffusion](https://mlsanity.com/paper/2310.08579/) (2023)
- [Animate Anyone 2: High-Fidelity Character Image Animation with Environment Affordance](https://mlsanity.com/paper/2502.06145/) (2025)
- [Coarse-to-Fine Latent Diffusion for Pose-Guided Person Image Synthesis](https://mlsanity.com/paper/2402.18078/) (2024)
- [Tunnel Try-on: Excavating Spatial-temporal Tunnels for High-quality Virtual Try-on in Videos](https://mlsanity.com/paper/2404.17571/) (2024)
- [VividPose: Advancing Stable Video Diffusion for Realistic Human Image Animation](https://mlsanity.com/paper/2405.18156/) (2024)
- [Learning Flow Fields in Attention for Controllable Person Image Generation](https://mlsanity.com/paper/2412.08486/) (2024)
- [Bidirectionally Deformable Motion Modulation For Video-based Human Pose Transfer](https://mlsanity.com/paper/2307.07754/) (2023)
- [HumanRefiner: Benchmarking Abnormal Human Generation and Refining with Coarse-to-fine Pose-Reversible Guidance](https://mlsanity.com/paper/2407.06937/) (2024)
- [Learning Disentangled Identifiers for Action-Customized Text-to-Image Generation](https://mlsanity.com/paper/2311.15841/) (2023)
- [View-decoupled Transformer for Person Re-identification under Aerial-ground Camera Network](https://mlsanity.com/paper/2403.14513/) (2024)
- [Towards Effective Usage of Human-Centric Priors in Diffusion Models for Text-based Human Image Generation](https://mlsanity.com/paper/2403.05239/) (2024)
- [Towards Multiple Character Image Animation Through Enhancing Implicit Decoupling](https://mlsanity.com/paper/2406.03035/) (2024)
- [GRPose: Learning Graph Relations for Human Image Generation with Pose Priors](https://mlsanity.com/paper/2408.16540/) (2024)
- [Tackling the Singularities at the Endpoints of Time Intervals in Diffusion Models](https://mlsanity.com/paper/2403.08381/) (2024)
- [UPGPT: Universal Diffusion Model for Person Image Generation, Editing and Pose Transfer](https://mlsanity.com/paper/2304.08870/) (2023)
- [UniHuman: A Unified Model for Editing Human Images in the Wild](https://mlsanity.com/paper/2312.14985/) (2023)
- [Cross-view Masked Diffusion Transformers for Person Image Synthesis](https://mlsanity.com/paper/2402.01516/) (2024)
- [Stable-Pose: Leveraging Transformers for Pose-Guided Text-to-Image Generation](https://mlsanity.com/paper/2406.02485/) (2024)
- [HumanDiffusion: a Coarse-to-Fine Alignment Diffusion Framework for Controllable Text-Driven Person Image Generation](https://mlsanity.com/paper/2211.06235/) (2022)
- [Human Image Generation: A Comprehensive Survey](https://mlsanity.com/paper/2212.08896/) (2022)
- [Animate-X++: Universal Character Image Animation with Dynamic Backgrounds](https://mlsanity.com/paper/2508.09454/) (2025)
- [UnitedHuman: Harnessing Multi-Source Data for High-Resolution Human Generation](https://mlsanity.com/paper/2309.14335/) (2023)
- [BeyondScene: Higher-Resolution Human-Centric Scene Generation With Pretrained Diffusion](https://mlsanity.com/paper/2404.04544/) (2024)

## Abstract (arXiv)

Pose Guided Person Image Generation (PGPIG) is the task of transforming a person image from the source pose to a given target pose. Most of the existing methods only focus on the ill-posed source-to-target task and fail to capture reasonable texture mapping. To address this problem, we propose a novel Dual-task Pose Transformer Network (DPTN), which introduces an auxiliary task (i.e., source-to-source task) and exploits the dual-task correlation to promote the performance of PGPIG. The DPTN is of a Siamese structure, containing a source-to-source self-reconstruction branch, and a transformation branch for source-to-target generation. By sharing partial weights between them, the knowledge learned by the source-to-source task can effectively assist the source-to-target learning. Furthermore, we bridge the two branches with a proposed Pose Transformer Module (PTM) to adaptively explore the correlation between features from dual tasks. Such correlation can establish the fine-grained mapping of all the pixels between the sources and the targets, and promote the source texture transmission to enhance the details of the generated target images. Extensive experiments show that our DPTN outperforms state-of-the-arts in terms of both PSNR and LPIPS. In addition, our DPTN only contains 9.79 million parameters, which is significantly smaller than other approaches. Our code is available at: https://github.com/PangzeCheung/Dual-task-Pose-Transformer-Network.