# Pose with Style: Detail-Preserving Pose-Guided Image Synthesis with Conditional StyleGAN

arXiv: https://arxiv.org/abs/2109.06166 · HTML: https://mlsanity.com/paper/2109.06166/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Builds on 66 of its 83 references that are in the map.
- First posted on arXiv 13 Sep 2021.

## Builds on (most cited first, 30 of 66)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [Nerfies: Deformable Neural Radiance Fields](https://mlsanity.com/paper/2011.12948/) (2020)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks](https://mlsanity.com/paper/1604.02878/) (2016)
- [Self-Attention Generative Adversarial Networks](https://mlsanity.com/paper/1805.08318/) (2018)
- [Multimodal Unsupervised Image-to-Image Translation](https://mlsanity.com/paper/1804.04732/) (2018)
- [Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes](https://mlsanity.com/paper/2011.13084/) (2020)
- [SphereFace: Deep Hypersphere Embedding for Face Recognition](https://mlsanity.com/paper/1704.08063/) (2017)
- [GANSpace: Discovering Interpretable GAN Controls](https://mlsanity.com/paper/2004.02546/) (2020)
- [Free-Form Image Inpainting with Gated Convolution](https://mlsanity.com/paper/1806.03589/) (2018)
- [DensePose: Dense Human Pose Estimation In The Wild](https://mlsanity.com/paper/1802.00434/) (2018)
- [Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstruction](https://mlsanity.com/paper/2012.03065/) (2020)
- [One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing](https://mlsanity.com/paper/2011.15126/) (2020)
- [Dynamic View Synthesis from Dynamic Monocular Video](https://mlsanity.com/paper/2105.06468/) (2021)
- [Space-time Neural Irradiance Fields for Free-Viewpoint Video](https://mlsanity.com/paper/2011.12950/) (2020)
- [Closed-Form Factorization of Latent Semantics in GANs](https://mlsanity.com/paper/2007.06600/) (2020)
- [Video-to-Video Synthesis](https://mlsanity.com/paper/1808.06601/) (2018)
- [StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flows](https://mlsanity.com/paper/2008.02401/) (2020)
- [Everybody Dance Now](https://mlsanity.com/paper/1808.07371/) (2018)
- [Diverse Image-to-Image Translation via Disentangled Representations](https://mlsanity.com/paper/1808.00948/) (2018)
- [Few-Shot Adversarial Learning of Realistic Neural Talking Head Models](https://mlsanity.com/paper/1905.08233/) (2019)
- [State of the Art on Neural Rendering](https://mlsanity.com/paper/2004.03805/) (2020)
- [StyleRig: Rigging StyleGAN for 3D Control over Portrait Images](https://mlsanity.com/paper/2004.00121/) (2020)
- [Deep Video Portraits](https://mlsanity.com/paper/1805.11714/) (2018)

## Built on by (most cited first, 30 of 45)

- [DreamPose: Fashion Image-to-Video Synthesis via Stable Diffusion](https://mlsanity.com/paper/2304.06025/) (2023)
- [StyleGAN-Human: A Data-Centric Odyssey of Human Generation](https://mlsanity.com/paper/2204.11823/) (2022)
- [Style-Based Global Appearance Flow for Virtual Try-On](https://mlsanity.com/paper/2204.01046/) (2022)
- [gDNA: Towards Generative Detailed Neural Avatars](https://mlsanity.com/paper/2201.04123/) (2022)
- [Neural Texture Extraction and Distribution for Controllable Person Image Synthesis](https://mlsanity.com/paper/2204.06160/) (2022)
- [AG3D: Learning to Generate 3D Avatars from 2D Image Collections](https://mlsanity.com/paper/2305.02312/) (2023)
- [Text2Performer: Text-Driven Human Video Generation](https://mlsanity.com/paper/2304.08483/) (2023)
- [Putting People in Their Place: Affordance-Aware Human Insertion into Scenes](https://mlsanity.com/paper/2304.14406/) (2023)
- [CM-GAN: Image Inpainting with Cascaded Modulation GAN and Object-Aware Training](https://mlsanity.com/paper/2203.11947/) (2022)
- [InsetGAN for Full-Body Image Generation](https://mlsanity.com/paper/2203.07293/) (2022)
- [Single-Image 3D Human Digitization with Shape-Guided Diffusion](https://mlsanity.com/paper/2311.09221/) (2023)
- [Self-Distilled StyleGAN: Towards Generation from Internet Photos](https://mlsanity.com/paper/2202.12211/) (2022)
- [TransEditor: Transformer-Based Dual-Space GAN for Highly Controllable Facial Editing](https://mlsanity.com/paper/2203.17266/) (2022)
- [BlobGAN: Spatially Disentangled Scene Representations](https://mlsanity.com/paper/2205.02837/) (2022)
- [State-of-the-Art in the Architecture, Methods and Applications of StyleGAN](https://mlsanity.com/paper/2202.14020/) (2022)
- [Towards Garment Sewing Pattern Reconstruction from a Single Image](https://mlsanity.com/paper/2311.04218/) (2023)
- [Street TryOn: Learning In-the-Wild Virtual Try-On from Unpaired Person Images](https://mlsanity.com/paper/2311.16094/) (2023)
- [HumanGen: Generating Human Radiance Fields with Explicit Priors](https://mlsanity.com/paper/2212.05321/) (2022)
- [Text2Human: Text-Driven Controllable Human Image Generation](https://mlsanity.com/paper/2205.15996/) (2022)
- [Rewriting Geometric Rules of a GAN](https://mlsanity.com/paper/2207.14288/) (2022)
- [3DHumanGAN: 3D-Aware Human Image Generation with 3D Pose Mapping](https://mlsanity.com/paper/2212.07378/) (2022)
- [Realistic Full-Body Anonymization with Surface-Guided GANs](https://mlsanity.com/paper/2201.02193/) (2022)
- [HS-Diffusion: Semantic-Mixing Diffusion for Head Swapping](https://mlsanity.com/paper/2212.06458/) (2022)
- [Learning Motion-Dependent Appearance for High-Fidelity Rendering of Dynamic Humans from a Single Camera](https://mlsanity.com/paper/2203.12780/) (2022)
- [Human Image Generation: A Comprehensive Survey](https://mlsanity.com/paper/2212.08896/) (2022)
- [Weakly Supervised High-Fidelity Clothing Model Generation](https://mlsanity.com/paper/2112.07200/) (2021)
- [FashionTex: Controllable Virtual Try-on with Text and Texture](https://mlsanity.com/paper/2305.04451/) (2023)
- [UnitedHuman: Harnessing Multi-Source Data for High-Resolution Human Generation](https://mlsanity.com/paper/2309.14335/) (2023)
- [Generating Texture for 3D Human Avatar from a Single Image using Sampling and Refinement Networks](https://mlsanity.com/paper/2305.00936/) (2023)
- [Content-Based Search for Deep Generative Models](https://mlsanity.com/paper/2210.03116/) (2022)

## Abstract (arXiv)

We present an algorithm for re-rendering a person from a single image under arbitrary poses. Existing methods often have difficulties in hallucinating occluded contents photo-realistically while preserving the identity and fine details in the source image. We first learn to inpaint the correspondence field between the body surface texture and the source image with a human body symmetry prior. The inpainted correspondence field allows us to transfer/warp local features extracted from the source to the target view even under large pose changes. Directly mapping the warped local features to an RGB image using a simple CNN decoder often leads to visible artifacts. Thus, we extend the StyleGAN generator so that it takes pose as input (for controlling poses) and introduces a spatially varying modulation for the latent space using the warped local features (for controlling appearances). We show that our method compares favorably against the state-of-the-art algorithms in both quantitative evaluation and visual comparison.