# Towards Using Clothes Style Transfer for Scenario-aware Person Video Generation

arXiv: https://arxiv.org/abs/2110.11894 · HTML: https://mlsanity.com/paper/2110.11894/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 31 of its 36 references that are in the map.
- First posted on arXiv 14 Oct 2021.

## Builds on (most cited first, 30 of 31)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Towards Accurate Generative Models of Video: A New Metric & Challenges](https://mlsanity.com/paper/1812.01717/) (2018)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1611.04076/) (2016)
- [First Order Motion Model for Image Animation](https://mlsanity.com/paper/2003.00196/) (2020)
- [Everybody Dance Now](https://mlsanity.com/paper/1808.07371/) (2018)
- [VITON: An Image-based Virtual Try-on Network](https://mlsanity.com/paper/1711.08447/) (2017)
- [Animating Arbitrary Objects via Deep Motion Transfer](https://mlsanity.com/paper/1812.08861/) (2018)
- [Few-shot Video-to-Video Synthesis](https://mlsanity.com/paper/1910.12713/) (2019)
- [Real-Time High-Resolution Background Matting](https://mlsanity.com/paper/2012.07810/) (2020)
- [Deformable GANs for Pose-based Human Image Generation](https://mlsanity.com/paper/1801.00055/) (2017)
- [Controllable Person Image Synthesis with Attribute-Decomposed GAN](https://mlsanity.com/paper/2003.12267/) (2020)
- [Towards Photo-Realistic Virtual Try-On by Adaptively GeneratingleftrightarrowPreserving Image Content](https://mlsanity.com/paper/2003.05863/) (2020)
- [Progressive Pose Attention Transfer for Person Image Generation](https://mlsanity.com/paper/1904.03349/) (2019)
- [A Variational U-Net for Conditional Appearance and Shape Generation](https://mlsanity.com/paper/1804.04694/) (2018)
- [DwNet: Dense warp-based network for pose-guided human video generation](https://mlsanity.com/paper/1910.09139/) (2019)
- [Revisiting the Sibling Head in Object Detector](https://mlsanity.com/paper/2003.07540/) (2020)
- [StylePeople: A Generative Model of Fullbody Human Avatars](https://mlsanity.com/paper/2104.08363/) (2021)
- [Unsupervised Person Image Synthesis in Arbitrary Poses](https://mlsanity.com/paper/1809.10280/) (2018)
- [Deep Spatial Transformation for Pose-Guided Person Image Generation and Animation](https://mlsanity.com/paper/2008.12606/) (2020)
- [Detailed Garment Recovery from a Single-View Image](https://mlsanity.com/paper/1608.01250/) (2016)

## Abstract (arXiv)

Clothes style transfer for person video generation is a challenging task, due to drastic variations of intra-person appearance and video scenarios. To tackle this problem, most recent AdaIN-based architectures are proposed to extract clothes and scenario features for generation. However, these approaches suffer from being short of fine-grained details and are prone to distort the origin person. To further improve the generation performance, we propose a novel framework with disentangled multi-branch encoders and a shared decoder. Moreover, to pursue the strong video spatio-temporal consistency, an inner-frame discriminator is delicately designed with input being cross-frame difference. Besides, the proposed framework possesses the property of scenario adaptation. Extensive experiments on the TEDXPeople benchmark demonstrate the superiority of our method over state-of-the-art approaches in terms of image quality and video coherence.