# Self-supervised Correlation Mining Network for Person Image Generation

arXiv: https://arxiv.org/abs/2111.13307 · HTML: https://mlsanity.com/paper/2111.13307/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 41 references that are in the map.
- First posted on arXiv 26 Nov 2021.

## Builds on (most cited first, 30 of 33)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [SEAN: Image Synthesis with Semantic Region-Adaptive Normalization](https://mlsanity.com/paper/1911.12861/) (2019)
- [Pose Guided Person Image Generation](https://mlsanity.com/paper/1705.09368/) (2017)
- [Deformable GANs for Pose-based Human Image Generation](https://mlsanity.com/paper/1801.00055/) (2017)
- [Controllable Person Image Synthesis with Attribute-Decomposed GAN](https://mlsanity.com/paper/2003.12267/) (2020)
- [Progressive Pose Attention Transfer for Person Image Generation](https://mlsanity.com/paper/1904.03349/) (2019)
- [A Variational U-Net for Conditional Appearance and Shape Generation](https://mlsanity.com/paper/1804.04694/) (2018)
- [Visual Attribute Transfer through Deep Image Analogy](https://mlsanity.com/paper/1705.01088/) (2017)
- [Cross-domain Correspondence Learning for Exemplar-based Image Translation](https://mlsanity.com/paper/2004.05571/) (2020)
- [Look into Person: Self-supervised Structure-sensitive Learning and A New Benchmark for Human Parsing](https://mlsanity.com/paper/1703.05446/) (2017)
- [Dense Intrinsic Appearance Flow for Human Pose Transfer](https://mlsanity.com/paper/1903.11326/) (2019)
- [Deep Image Spatial Transformation for Person Image Generation](https://mlsanity.com/paper/2003.00696/) (2020)
- [Disentangled Person Image Generation](https://mlsanity.com/paper/1712.02621/) (2017)
- [Deep Exemplar-based Colorization](https://mlsanity.com/paper/1807.06587/) (2018)
- [XingGAN for Person Image Generation](https://mlsanity.com/paper/2007.09278/) (2020)
- [PISE: Person Image Synthesis and Editing with Decoupled GAN](https://mlsanity.com/paper/2103.04023/) (2021)
- [Soft-Gated Warping-GAN for Pose-Guided Person Image Synthesis](https://mlsanity.com/paper/1810.11610/) (2018)
- [Inter-Region Affinity Distillation for Road Marking Segmentation](https://mlsanity.com/paper/2004.05304/) (2020)
- [Unsupervised Person Image Synthesis in Arbitrary Poses](https://mlsanity.com/paper/1809.10280/) (2018)
- [Learning Context Graph for Person Search](https://mlsanity.com/paper/1904.01830/) (2019)
- [Reference-Based Sketch Image Colorization using Augmented-Self Reference and Dense Semantic Correspondence](https://mlsanity.com/paper/2005.05207/) (2020)
- [Person Re-identification with Deep Similarity-Guided Graph Neural Network](https://mlsanity.com/paper/1807.09975/) (2018)
- [Adaptive Graph Representation Learning for Video Person Re-identification](https://mlsanity.com/paper/1909.02240/) (2019)

## Built on by (most cited first, 9 of 9)

- [EmotionGesture: Audio-Driven Diverse Emotional Co-Speech 3D Gesture Generation](https://mlsanity.com/paper/2305.18891/) (2023)
- [Diverse 3D Hand Gesture Prediction from Body Dynamics by Bilateral Hand Disentanglement](https://mlsanity.com/paper/2303.01765/) (2023)
- [Weakly-Supervised Emotion Transition Learning for Diverse 3D Co-speech Gesture Generation](https://mlsanity.com/paper/2311.17532/) (2023)
- [UniHuman: A Unified Model for Editing Human Images in the Wild](https://mlsanity.com/paper/2312.14985/) (2023)
- [HumanDiffusion: a Coarse-to-Fine Alignment Diffusion Framework for Controllable Text-Driven Person Image Generation](https://mlsanity.com/paper/2211.06235/) (2022)
- [Human Image Generation: A Comprehensive Survey](https://mlsanity.com/paper/2212.08896/) (2022)
- [Collecting The Puzzle Pieces: Disentangled Self-Driven Human Pose Transfer by Permuting Textures](https://mlsanity.com/paper/2210.01887/) (2022)
- [Enhanced Multi-Scale Cross-Attention for Person Image Generation](https://mlsanity.com/paper/2501.08900/) (2025)
- [Biphasic Face Photo-Sketch Synthesis via Semantic-Driven Generative Adversarial Network with Graph Representation Learning](https://mlsanity.com/paper/2201.01592/) (2022)

## Abstract (arXiv)

Person image generation aims to perform non-rigid deformation on source images, which generally requires unaligned data pairs for training. Recently, self-supervised methods express great prospects in this task by merging the disentangled representations for self-reconstruction. However, such methods fail to exploit the spatial correlation between the disentangled features. In this paper, we propose a Self-supervised Correlation Mining Network (SCM-Net) to rearrange the source images in the feature space, in which two collaborative modules are integrated, Decomposed Style Encoder (DSE) and Correlation Mining Module (CMM). Specifically, the DSE first creates unaligned pairs at the feature level. Then, the CMM establishes the spatial correlation field for feature rearrangement. Eventually, a translation module transforms the rearranged features to realistic results. Meanwhile, for improving the fidelity of cross-scale pose transformation, we propose a graph based Body Structure Retaining Loss (BSR Loss) to preserve reasonable body structures on half body to full body generation. Extensive experiments conducted on DeepFashion dataset demonstrate the superiority of our method compared with other supervised and unsupervised approaches. Furthermore, satisfactory results on face generation show the versatility of our method in other deformation tasks.