# Semantically Consistent Person Image Generation

arXiv: https://arxiv.org/abs/2302.14728 · HTML: https://mlsanity.com/paper/2302.14728/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 27 of its 35 references that are in the map.
- First posted on arXiv 28 Feb 2023.

## Builds on (most cited first, 27 of 27)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [Deep Learning for Person Re-identification: A Survey and Outlook](https://mlsanity.com/paper/2001.04193/) (2020)
- [VITON: An Image-based Virtual Try-on Network](https://mlsanity.com/paper/1711.08447/) (2017)
- [Pose Guided Person Image Generation](https://mlsanity.com/paper/1705.09368/) (2017)
- [Self-Correction for Human Parsing](https://mlsanity.com/paper/1910.09777/) (2019)
- [Deformable GANs for Pose-based Human Image Generation](https://mlsanity.com/paper/1801.00055/) (2017)
- [Progressive Pose Attention Transfer for Person Image Generation](https://mlsanity.com/paper/1904.03349/) (2019)
- [A Variational U-Net for Conditional Appearance and Shape Generation](https://mlsanity.com/paper/1804.04694/) (2018)
- [Dense Intrinsic Appearance Flow for Human Pose Transfer](https://mlsanity.com/paper/1903.11326/) (2019)
- [Disentangled Person Image Generation](https://mlsanity.com/paper/1712.02621/) (2017)
- [Synthesizing Images of Humans in Unseen Poses](https://mlsanity.com/paper/1804.07739/) (2018)
- [Dense Pose Transfer](https://mlsanity.com/paper/1809.01995/) (2018)
- [XingGAN for Person Image Generation](https://mlsanity.com/paper/2007.09278/) (2020)
- [Deep Human Parsing with Active Template Regression](https://mlsanity.com/paper/1503.02391/) (2015)
- [Bipartite Graph Reasoning GANs for Person Image Generation](https://mlsanity.com/paper/2008.04381/) (2020)
- [Text Guided Person Image Synthesis](https://mlsanity.com/paper/1904.05118/) (2019)
- [TIPS: Text-Induced Pose Synthesis](https://mlsanity.com/paper/2207.11718/) (2022)
- [Wish You Were Here: Context-Aware Human Generation](https://mlsanity.com/paper/2005.10663/) (2020)
- [Multi-scale Attention Guided Pose Transfer](https://mlsanity.com/paper/2202.06777/) (2022)
- [Scene Aware Person Image Generation through Global Contextual Conditioning](https://mlsanity.com/paper/2206.02717/) (2022)

## Abstract (arXiv)

We propose a data-driven approach for context-aware person image generation. Specifically, we attempt to generate a person image such that the synthesized instance can blend into a complex scene. In our method, the position, scale, and appearance of the generated person are semantically conditioned on the existing persons in the scene. The proposed technique is divided into three sequential steps. At first, we employ a Pix2PixHD model to infer a coarse semantic mask that represents the new person's spatial location, scale, and potential pose. Next, we use a data-centric approach to select the closest representation from a precomputed cluster of fine semantic masks. Finally, we adopt a multi-scale, attention-guided architecture to transfer the appearance attributes from an exemplar image. The proposed strategy enables us to synthesize semantically coherent realistic persons that can blend into an existing scene without altering the global context. We conclude our findings with relevant qualitative and quantitative evaluations.