# Learning Realistic Human Reposing using Cyclic Self-Supervision with 3D Shape, Pose, and Appearance Consistency

arXiv: https://arxiv.org/abs/2110.05458 · HTML: https://mlsanity.com/paper/2110.05458/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 50 references that are in the map.
- First posted on arXiv 11 Oct 2021.

## Builds on (most cited first, 30 of 38)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1812.08008/) (2018)
- [Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image](https://mlsanity.com/paper/1607.08128/) (2016)
- [Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loop](https://mlsanity.com/paper/1909.12828/) (2019)
- [DensePose: Dense Human Pose Estimation In The Wild](https://mlsanity.com/paper/1802.00434/) (2018)
- [Pose Guided Person Image Generation](https://mlsanity.com/paper/1705.09368/) (2017)
- [Deformable GANs for Pose-based Human Image Generation](https://mlsanity.com/paper/1801.00055/) (2017)
- [Liquid Warping GAN: A Unified Framework for Human Motion Imitation, Appearance Transfer and Novel View Synthesis](https://mlsanity.com/paper/1909.12224/) (2019)
- [Controllable Person Image Synthesis with Attribute-Decomposed GAN](https://mlsanity.com/paper/2003.12267/) (2020)
- [Progressive Pose Attention Transfer for Person Image Generation](https://mlsanity.com/paper/1904.03349/) (2019)
- [A Variational U-Net for Conditional Appearance and Shape Generation](https://mlsanity.com/paper/1804.04694/) (2018)
- [Cross-domain Correspondence Learning for Exemplar-based Image Translation](https://mlsanity.com/paper/2004.05571/) (2020)
- [DwNet: Dense warp-based network for pose-guided human video generation](https://mlsanity.com/paper/1910.09139/) (2019)
- [Dense Intrinsic Appearance Flow for Human Pose Transfer](https://mlsanity.com/paper/1903.11326/) (2019)
- [Deep Image Spatial Transformation for Person Image Generation](https://mlsanity.com/paper/2003.00696/) (2020)
- [Disentangled Person Image Generation](https://mlsanity.com/paper/1712.02621/) (2017)
- [Synthesizing Images of Humans in Unseen Poses](https://mlsanity.com/paper/1804.07739/) (2018)
- [Dense Pose Transfer](https://mlsanity.com/paper/1809.01995/) (2018)
- [XingGAN for Person Image Generation](https://mlsanity.com/paper/2007.09278/) (2020)
- [Pose Guided Human Video Generation](https://mlsanity.com/paper/1807.11152/) (2018)
- [Soft-Gated Warping-GAN for Pose-Guided Person Image Synthesis](https://mlsanity.com/paper/1810.11610/) (2018)
- [Neural Re-Rendering of Humans from a Single Image](https://mlsanity.com/paper/2101.04104/) (2021)
- [Unsupervised Person Image Synthesis in Arbitrary Poses](https://mlsanity.com/paper/1809.10280/) (2018)
- [On Aliased Resizing and Surprising Subtleties in GAN Evaluation](https://mlsanity.com/paper/2104.11222/) (2021)

## Built on by (most cited first, 15 of 15)

- [HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Video](https://mlsanity.com/paper/2201.04127/) (2022)
- [NeuMan: Neural Human Radiance Field from a Single Video](https://mlsanity.com/paper/2203.12575/) (2022)
- [InsetGAN for Full-Body Image Generation](https://mlsanity.com/paper/2203.07293/) (2022)
- [Cross Attention Based Style Distribution for Controllable Person Image Synthesis](https://mlsanity.com/paper/2208.00712/) (2022)
- [3D-Aware Semantic-Guided Generative Model for Human Synthesis](https://mlsanity.com/paper/2112.01422/) (2021)
- [Learning Flow Fields in Attention for Controllable Person Image Generation](https://mlsanity.com/paper/2412.08486/) (2024)
- [3DHumanGAN: 3D-Aware Human Image Generation with 3D Pose Mapping](https://mlsanity.com/paper/2212.07378/) (2022)
- [SMPLer: Taming Transformers for Monocular 3D Human Shape and Pose Estimation](https://mlsanity.com/paper/2404.15276/) (2024)
- [Generalizable Neural Voxels for Fast Human Radiance Fields](https://mlsanity.com/paper/2303.15387/) (2023)
- [SCULPT: Shape-Conditioned Unpaired Learning of Pose-dependent Clothed and Textured Human Meshes](https://mlsanity.com/paper/2308.10638/) (2023)
- [Collecting The Puzzle Pieces: Disentangled Self-Driven Human Pose Transfer by Permuting Textures](https://mlsanity.com/paper/2210.01887/) (2022)
- [Total Selfie: Generating Full-Body Selfies](https://mlsanity.com/paper/2308.14740/) (2023)
- [Synthesizing Environment-Specific People in Photographs](https://mlsanity.com/paper/2312.14579/) (2023)
- [GaitMorph: Transforming Gait by Optimally Transporting Discrete Codes](https://mlsanity.com/paper/2307.14713/) (2023)
- [Pro-Pose: Unpaired Full-Body Portrait Synthesis via Canonical UV Maps](https://mlsanity.com/paper/2512.17143/) (2025)

## Abstract (arXiv)

Synthesizing images of a person in novel poses from a single image is a highly ambiguous task. Most existing approaches require paired training images; i.e. images of the same person with the same clothing in different poses. However, obtaining sufficiently large datasets with paired data is challenging and costly. Previous methods that forego paired supervision lack realism. We propose a self-supervised framework named SPICE (Self-supervised Person Image CrEation) that closes the image quality gap with supervised methods. The key insight enabling self-supervision is to exploit 3D information about the human body in several ways. First, the 3D body shape must remain unchanged when reposing. Second, representing body pose in 3D enables reasoning about self occlusions. Third, 3D body parts that are visible before and after reposing, should have similar appearance features. Once trained, SPICE takes an image of a person and generates a new image of that person in a new target pose. SPICE achieves state-of-the-art performance on the DeepFashion dataset, improving the FID score from 29.9 to 7.8 compared with previous unsupervised methods, and with performance similar to the state-of-the-art supervised method (6.4). SPICE also generates temporally coherent videos given an input image and a sequence of poses, despite being trained on static images only.