# StyleVideoGAN: A Temporal Generative Model using a Pretrained StyleGAN

arXiv: https://arxiv.org/abs/2107.07224 · HTML: https://mlsanity.com/paper/2107.07224/

## Key facts

- Cited by 47 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 38 references that are in the map.
- First posted on arXiv 15 Jul 2021.

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop](https://mlsanity.com/paper/1506.03365/) (2015)
- [Towards Accurate Generative Models of Video: A New Metric & Challenges](https://mlsanity.com/paper/1812.01717/) (2018)
- [Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation](https://mlsanity.com/paper/2008.00951/) (2020)
- [Interpreting the Latent Space of GANs for Semantic Face Editing](https://mlsanity.com/paper/1907.10786/) (2019)
- [Wasserstein GAN](https://mlsanity.com/paper/1701.07875/) (2017)
- [GANSpace: Discovering Interpretable GAN Controls](https://mlsanity.com/paper/2004.02546/) (2020)
- [MoCoGAN: Decomposing Motion and Content for Video Generation](https://mlsanity.com/paper/1707.04993/) (2017)
- [Generating Videos with Scene Dynamics](https://mlsanity.com/paper/1609.02612/) (2016)
- [GAN Inversion: A Survey](https://mlsanity.com/paper/2101.05278/) (2021)
- [StyleRig: Rigging StyleGAN for 3D Control over Portrait Images](https://mlsanity.com/paper/2004.00121/) (2020)
- [Temporal Generative Adversarial Nets with Singular Value Clipping](https://mlsanity.com/paper/1611.06624/) (2016)
- [Scaling Autoregressive Video Models](https://mlsanity.com/paper/1906.02634/) (2019)
- [A Good Image Generator Is What You Need for High-Resolution Video Synthesis](https://mlsanity.com/paper/2104.15069/) (2021)
- [Unsupervised Learning of Disentangled Representations from Video](https://mlsanity.com/paper/1705.10915/) (2017)
- [Train Sparsely, Generate Densely: Memory-efficient Unsupervised Training of High-resolution Temporal GAN](https://mlsanity.com/paper/1811.09245/) (2018)
- [Transforming and Projecting Images into Class-conditional Generative Networks](https://mlsanity.com/paper/2005.01703/) (2020)
- [Generating High Fidelity Images with Subscale Pixel Networks and Multidimensional Upscaling](https://mlsanity.com/paper/1812.01608/) (2018)
- [Playable Video Generation](https://mlsanity.com/paper/2101.12195/) (2021)
- [Lower Dimensional Kernels for Video Discriminators](https://mlsanity.com/paper/1912.08860/) (2019)
- [Towards High Resolution Video Generation with Progressive Growing of Sliced Wasserstein GANs](https://mlsanity.com/paper/1810.02419/) (2018)
- [InMoDeGAN: Interpretable Motion Decomposition Generative Adversarial Network for Video Generation](https://mlsanity.com/paper/2101.03049/) (2021)
- [Markov Decision Process for Video Generation](https://mlsanity.com/paper/1909.12400/) (2019)

## Built on by (most cited first, 30 of 47)

- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2304.08818/) (2023)
- [Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Driving](https://mlsanity.com/paper/2311.17918/) (2023)
- [StyleGAN-V: A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN2](https://mlsanity.com/paper/2112.14683/) (2021)
- [Generating Videos with Dynamics-aware Implicit Generative Adversarial Networks](https://mlsanity.com/paper/2202.10571/) (2022)
- [Video Probabilistic Diffusion Models in Projected Latent Space](https://mlsanity.com/paper/2302.07685/) (2023)
- [StyleHEAT: One-Shot High-Resolution Editable Talking Face Generation via Pre-trained StyleGAN](https://mlsanity.com/paper/2203.04036/) (2022)
- [ConsistI2V: Enhancing Visual Consistency for Image-to-Video Generation](https://mlsanity.com/paper/2402.04324/) (2024)
- [Generating Long Videos of Dynamic Scenes](https://mlsanity.com/paper/2206.03429/) (2022)
- [InfiniteNature-Zero: Learning Perpetual View Generation of Natural Scenes from Single Images](https://mlsanity.com/paper/2207.11148/) (2022)
- [VIDM: Video Implicit Diffusion Models](https://mlsanity.com/paper/2212.00235/) (2022)
- [Stitch it in Time: GAN-Based Facial Editing of Real Videos](https://mlsanity.com/paper/2201.08361/) (2022)
- [AR-Diffusion: Asynchronous Video Generation with Auto-Regressive Diffusion](https://mlsanity.com/paper/2503.07418/) (2025)
- [Persistent Nature: A Generative Model of Unbounded 3D Worlds](https://mlsanity.com/paper/2303.13515/) (2023)
- [Efficient Video Diffusion Models via Content-Frame Motion-Latent Decomposition](https://mlsanity.com/paper/2403.14148/) (2024)
- [Sound-Guided Semantic Video Generation](https://mlsanity.com/paper/2204.09273/) (2022)
- [Dysen-VDM: Empowering Dynamics-aware Text-to-Video Diffusion with LLMs](https://mlsanity.com/paper/2308.13812/) (2023)
- [Picture that Sketch: Photorealistic Image Generation from Abstract Sketches](https://mlsanity.com/paper/2303.11162/) (2023)
- [PV3D: A 3D Generative Model for Portrait Video Generation](https://mlsanity.com/paper/2212.06384/) (2022)
- [VToonify: Controllable High-Resolution Portrait Video Style Transfer](https://mlsanity.com/paper/2209.11224/) (2022)
- [StyleInV: A Temporal Style Modulated Inversion Network for Unconditional Video Generation](https://mlsanity.com/paper/2308.16909/) (2023)
- [3D-Aware Video Generation](https://mlsanity.com/paper/2206.14797/) (2022)
- [Talking Head from Speech Audio using a Pre-trained Image Generator](https://mlsanity.com/paper/2209.04252/) (2022)
- [VIVE3D: Viewpoint-Independent Video Editing using 3D-Aware GANs](https://mlsanity.com/paper/2303.15893/) (2023)
- [StyleTalker: One-shot Style-based Audio-driven Talking Head Video Generation](https://mlsanity.com/paper/2208.10922/) (2022)
- [I4VGen: Image as Free Stepping Stone for Text-to-Video Generation](https://mlsanity.com/paper/2406.02230/) (2024)
- [MALT Diffusion: Memory-Augmented Latent Transformers for Any-Length Video Generation](https://mlsanity.com/paper/2502.12632/) (2025)
- [RIGID: Recurrent GAN Inversion and Editing of Real Face Videos](https://mlsanity.com/paper/2308.06097/) (2023)
- [Towards Smooth Video Composition](https://mlsanity.com/paper/2212.07413/) (2022)
- [StyleFaceV: Face Video Generation via Decomposing and Recomposing Pretrained StyleGAN3](https://mlsanity.com/paper/2208.07862/) (2022)

## Abstract (arXiv)

Generative adversarial models (GANs) continue to produce advances in terms of the visual quality of still images, as well as the learning of temporal correlations. However, few works manage to combine these two interesting capabilities for the synthesis of video content: Most methods require an extensive training dataset to learn temporal correlations, while being rather limited in the resolution and visual quality of their output. We present a novel approach to the video synthesis problem that helps to greatly improve visual quality and drastically reduce the amount of training data and resources necessary for generating videos. Our formulation separates the spatial domain, in which individual frames are synthesized, from the temporal domain, in which motion is generated. For the spatial domain we use a pre-trained StyleGAN network, the latent space of which allows control over the appearance of the objects it was trained for. The expressive power of this model allows us to embed our training videos in the StyleGAN latent space. Our temporal architecture is then trained not on sequences of RGB frames, but on sequences of StyleGAN latent codes. The advantageous properties of the StyleGAN space simplify the discovery of temporal correlations. We demonstrate that it suffices to train our temporal architecture on only 10 minutes of footage of 1 subject for about 6 hours. After training, our model can not only generate new portrait videos for the training subject, but also for any random subject which can be embedded in the StyleGAN space.