# MoStGAN-V: Video Generation with Temporal Motion Styles

arXiv: https://arxiv.org/abs/2304.02777 · HTML: https://mlsanity.com/paper/2304.02777/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 45 references that are in the map.
- First posted on arXiv 5 Apr 2023.

## Builds on (most cited first, 30 of 34)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [Video Diffusion Models](https://mlsanity.com/paper/2204.03458/) (2022)
- [Towards Accurate Generative Models of Video: A New Metric & Challenges](https://mlsanity.com/paper/1812.01717/) (2018)
- [Training Generative Adversarial Networks with Limited Data](https://mlsanity.com/paper/2006.06676/) (2020)
- [HyperNetworks](https://mlsanity.com/paper/1609.09106/) (2016)
- [CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformers](https://mlsanity.com/paper/2205.15868/) (2022)
- [Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space?](https://mlsanity.com/paper/1904.03189/) (2019)
- [VideoGPT: Video Generation using VQ-VAE and Transformers](https://mlsanity.com/paper/2104.10157/) (2021)
- [MoCoGAN: Decomposing Motion and Content for Video Generation](https://mlsanity.com/paper/1707.04993/) (2017)
- [Closed-Form Factorization of Latent Semantics in GANs](https://mlsanity.com/paper/2007.06600/) (2020)
- [Generating Videos with Scene Dynamics](https://mlsanity.com/paper/1609.02612/) (2016)
- [StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generation](https://mlsanity.com/paper/2011.12799/) (2020)
- [StyleGAN-V: A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN2](https://mlsanity.com/paper/2112.14683/) (2021)
- [Continual learning with hypernetworks](https://mlsanity.com/paper/1906.00695/) (2019)
- [CelebV-HQ: A Large-Scale Video Facial Attributes Dataset](https://mlsanity.com/paper/2207.12393/) (2022)
- [Long Video Generation with Time-Agnostic VQGAN and Time-Sensitive Transformer](https://mlsanity.com/paper/2204.03638/) (2022)
- [Diffusion Probabilistic Modeling for Video Generation](https://mlsanity.com/paper/2203.09481/) (2022)
- [Temporal Generative Adversarial Nets with Singular Value Clipping](https://mlsanity.com/paper/1611.06624/) (2016)
- [Generating Videos with Dynamics-aware Implicit Generative Adversarial Networks](https://mlsanity.com/paper/2202.10571/) (2022)
- [Latent Image Animator: Learning to Animate Images via Latent Space Navigation](https://mlsanity.com/paper/2203.09043/) (2022)
- [A Good Image Generator Is What You Need for High-Resolution Video Synthesis](https://mlsanity.com/paper/2104.15069/) (2021)
- [Adversarial Generation of Continuous Images](https://mlsanity.com/paper/2011.12026/) (2020)
- [Train Sparsely, Generate Densely: Memory-efficient Unsupervised Training of High-resolution Temporal GAN](https://mlsanity.com/paper/1811.09245/) (2018)
- [FaceForensics: A Large-scale Video Dataset for Forgery Detection in Human Faces](https://mlsanity.com/paper/1803.09179/) (2018)
- [Generating Long Videos of Dynamic Scenes](https://mlsanity.com/paper/2206.03429/) (2022)
- [Learning Deep Disentangled Embeddings with the F-Statistic Loss](https://mlsanity.com/paper/1802.05312/) (2018)
- [Learning feed-forward one-shot learners](https://mlsanity.com/paper/1606.05233/) (2016)

## Built on by (most cited first, 30 of 37)

- [Latte: Latent Diffusion Transformer for Video Generation](https://mlsanity.com/paper/2401.03048/) (2024)
- [Show-1: Marrying Pixel and Latent Diffusion Models for Text-to-Video Generation](https://mlsanity.com/paper/2309.15818/) (2023)
- [Preserve Your Own Correlation: A Noise Prior for Video Diffusion Models](https://mlsanity.com/paper/2305.10474/) (2023)
- [DreamVideo: Composing Your Dream Videos with Customized Subject and Motion](https://mlsanity.com/paper/2312.04433/) (2023)
- [MotionDirector: Motion Customization of Text-to-Video Diffusion Models](https://mlsanity.com/paper/2310.08465/) (2023)
- [OmniTokenizer: A Joint Image-Video Tokenizer for Visual Generation](https://mlsanity.com/paper/2406.09399/) (2024)
- [Snap Video: Scaled Spatiotemporal Transformers for Text-to-Video Synthesis](https://mlsanity.com/paper/2402.14797/) (2024)
- [DeMamba: AI-Generated Video Detection on Million-Scale GenVideo Benchmark](https://mlsanity.com/paper/2405.19707/) (2024)
- [Endora: Video Generation Models as Endoscopy Simulators](https://mlsanity.com/paper/2403.11050/) (2024)
- [AR-Diffusion: Asynchronous Video Generation with Auto-Regressive Diffusion](https://mlsanity.com/paper/2503.07418/) (2025)
- [On the Content Bias in Fr'echet Video Distance](https://mlsanity.com/paper/2404.12391/) (2024)
- [Hierarchical Spatio-temporal Decoupling for Text-to-Video Generation](https://mlsanity.com/paper/2312.04483/) (2023)
- [Moonshot: Towards Controllable Video Generation and Editing with Multimodal Conditions](https://mlsanity.com/paper/2401.01827/) (2024)
- [LEO: Generative Latent Image Animator for Human Video Synthesis](https://mlsanity.com/paper/2305.03989/) (2023)
- [Taming Teacher Forcing for Masked Autoregressive Video Generation](https://mlsanity.com/paper/2501.12389/) (2025)
- [Matten: Video Generation with Mamba-Attention](https://mlsanity.com/paper/2405.03025/) (2024)
- [Scaling Diffusion Mamba with Bidirectional SSMs for Efficient Image and Video Generation](https://mlsanity.com/paper/2405.15881/) (2024)
- [I4VGen: Image as Free Stepping Stone for Text-to-Video Generation](https://mlsanity.com/paper/2406.02230/) (2024)
- [Denoising with a Joint-Embedding Predictive Architecture](https://mlsanity.com/paper/2410.03755/) (2024)
- [Neural Residual Diffusion Models for Deep Scalable Vision Generation](https://mlsanity.com/paper/2406.13215/) (2024)
- [Optical Flow Representation Alignment Mamba Diffusion Model for Medical Video Generation](https://mlsanity.com/paper/2411.01647/) (2024)
- [Preserving Forgery Artifacts: AI-Generated Video Detection at Native Scale](https://mlsanity.com/paper/2604.04634/) (2026)
- [JointTuner: Appearance-Motion Adaptive Joint Training for Customized Video Generation](https://mlsanity.com/paper/2503.23951/) (2025)
- [Rethinking Human Evaluation Protocol for Text-to-Video Models: Enhancing Reliability,Reproducibility, and Practicality](https://mlsanity.com/paper/2406.08845/) (2024)
- [Separate Motion from Appearance: Customizing Motion via Customizing Text-to-Video Diffusion Models](https://mlsanity.com/paper/2501.16714/) (2025)
- [FEAT: Full-Dimensional Efficient Attention Transformer for Medical Video Generation](https://mlsanity.com/paper/2506.04956/) (2025)
- [GenDeF: Learning Generative Deformation Field for Video Generation](https://mlsanity.com/paper/2312.04561/) (2023)
- [DREAM: Improving Video-Text Retrieval Through Relevance-Based Augmentation Using Large Foundation Models](https://mlsanity.com/paper/2404.05083/) (2024)
- [Video to Video Generative Adversarial Network for Few-shot Learning Based on Policy Gradient](https://mlsanity.com/paper/2410.20657/) (2024)
- [CHAIN: Enhancing Generalization in Data-Efficient GANs via lipsCHitz continuity constrAIned Normalization](https://mlsanity.com/paper/2404.00521/) (2024)

## Abstract (arXiv)

Video generation remains a challenging task due to spatiotemporal complexity and the requirement of synthesizing diverse motions with temporal consistency. Previous works attempt to generate videos in arbitrary lengths either in an autoregressive manner or regarding time as a continuous signal. However, they struggle to synthesize detailed and diverse motions with temporal coherence and tend to generate repetitive scenes after a few time steps. In this work, we argue that a single time-agnostic latent vector of style-based generator is insufficient to model various and temporally-consistent motions. Hence, we introduce additional time-dependent motion styles to model diverse motion patterns. In addition, a Motion Style Attention modulation mechanism, dubbed as MoStAtt, is proposed to augment frames with vivid dynamics for each specific scale (i.e., layer), which assigns attention score for each motion style w.r.t deconvolution filter weights in the target synthesis layer and softly attends different motion styles for weight modulation. Experimental results show our model achieves state-of-the-art performance on four unconditional 2562 video synthesis benchmarks trained with only 3 frames per clip and produces better qualitative results with respect to dynamic motions. Code and videos have been made available at https://github.com/xiaoqian-shen/MoStGAN-V.