# TemporalUV: Capturing Loose Clothing with Temporally Coherent UV Coordinates

arXiv: https://arxiv.org/abs/2204.03671 · HTML: https://mlsanity.com/paper/2204.03671/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 49 references that are in the map.
- First posted on arXiv 7 Apr 2022.

## Builds on (most cited first, 30 of 35)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [First Order Motion Model for Image Animation](https://mlsanity.com/paper/2003.00196/) (2020)
- [Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image](https://mlsanity.com/paper/1607.08128/) (2016)
- [VIBE: Video Inference for Human Body Pose and Shape Estimation](https://mlsanity.com/paper/1912.05656/) (2019)
- [DensePose: Dense Human Pose Estimation In The Wild](https://mlsanity.com/paper/1802.00434/) (2018)
- [MoCoGAN: Decomposing Motion and Content for Video Generation](https://mlsanity.com/paper/1707.04993/) (2017)
- [Video-to-Video Synthesis](https://mlsanity.com/paper/1808.06601/) (2018)
- [Generating Videos with Scene Dynamics](https://mlsanity.com/paper/1609.02612/) (2016)
- [Everybody Dance Now](https://mlsanity.com/paper/1808.07371/) (2018)
- [Pose Guided Person Image Generation](https://mlsanity.com/paper/1705.09368/) (2017)
- [Learning to Estimate 3D Human Pose and Shape from a Single Color Image](https://mlsanity.com/paper/1805.04092/) (2018)
- [Animating Arbitrary Objects via Deep Motion Transfer](https://mlsanity.com/paper/1812.08861/) (2018)
- [Temporal Generative Adversarial Nets with Singular Value Clipping](https://mlsanity.com/paper/1611.06624/) (2016)
- [Deformable GANs for Pose-based Human Image Generation](https://mlsanity.com/paper/1801.00055/) (2017)
- [Liquid Warping GAN: A Unified Framework for Human Motion Imitation, Appearance Transfer and Novel View Synthesis](https://mlsanity.com/paper/1909.12224/) (2019)
- [DwNet: Dense warp-based network for pose-guided human video generation](https://mlsanity.com/paper/1910.09139/) (2019)
- [Synthesizing Images of Humans in Unseen Poses](https://mlsanity.com/paper/1804.07739/) (2018)
- [Coherent Online Video Style Transfer](https://mlsanity.com/paper/1703.09211/) (2017)
- [Recycle-GAN: Unsupervised Video Retargeting](https://mlsanity.com/paper/1808.05174/) (2018)
- [Dense Pose Transfer](https://mlsanity.com/paper/1809.01995/) (2018)
- [XingGAN for Person Image Generation](https://mlsanity.com/paper/2007.09278/) (2020)
- [Pose Guided Human Video Generation](https://mlsanity.com/paper/1807.11152/) (2018)
- [UV-GAN: Adversarial Facial UV Map Completion for Pose-invariant Face Recognition](https://mlsanity.com/paper/1712.04695/) (2017)
- [Unsupervised Person Image Synthesis in Arbitrary Poses](https://mlsanity.com/paper/1809.10280/) (2018)
- [Pose-Guided Human Animation from a Single Image in the Wild](https://mlsanity.com/paper/2012.03796/) (2020)
- [OSTeC: One-Shot Texture Completion](https://mlsanity.com/paper/2012.15370/) (2020)
- [Deep Video-Based Performance Cloning](https://mlsanity.com/paper/1808.06847/) (2018)
- [Implicit Feature Networks for Texture Completion from Partial 3D Data](https://mlsanity.com/paper/2009.09458/) (2020)
- [SimPose: Effectively Learning DensePose and Surface Normals of People from Simulated Data](https://mlsanity.com/paper/2007.15506/) (2020)
- [Slim DensePose: Thrifty Learning from Sparse Annotations and Motion Cues](https://mlsanity.com/paper/1906.05706/) (2019)

## Built on by (most cited first, 5 of 5)

- [Normal-guided Garment UV Prediction for Human Re-texturing](https://mlsanity.com/paper/2303.06504/) (2023)
- [Learning Garment DensePose for Robust Warping in Virtual Try-On](https://mlsanity.com/paper/2303.17688/) (2023)
- [A Generative Multi-Resolution Pyramid and Normal-Conditioning 3D Cloth Draping](https://mlsanity.com/paper/2311.02700/) (2023)
- [An Embeddable Implicit IUVD Representation for Part-based 3D Human Surface Reconstruction](https://mlsanity.com/paper/2401.16810/) (2024)
- [Pattern Guided UV Recovery for Realistic Video Garment Texturing](https://mlsanity.com/paper/2407.10137/) (2024)

## Abstract (arXiv)

We propose a novel approach to generate temporally coherent UV coordinates for loose clothing. Our method is not constrained by human body outlines and can capture loose garments and hair. We implemented a differentiable pipeline to learn UV mapping between a sequence of RGB inputs and textures via UV coordinates. Instead of treating the UV coordinates of each frame separately, our data generation approach connects all UV coordinates via feature matching for temporal stability. Subsequently, a generative model is trained to balance the spatial quality and temporal stability. It is driven by supervised and unsupervised losses in both UV and image spaces. Our experiments show that the trained models output high-quality UV coordinates and generalize to new poses. Once a sequence of UV coordinates has been inferred by our model, it can be used to flexibly synthesize new looks and modified visual styles. Compared to existing methods, our approach reduces the computational workload to animate new outfits by several orders of magnitude.