mlsanityData to 14 Oct 2026

VidStyleODE: Disentangled Video Editing via StyleGAN and NeuralODEs

arXiv 2304.06020 ↗First posted 12 Apr 2023cs.CVcited by 3

Not on the map. Why?

Abstract (arXiv)

We propose VidStyleODE, a spatiotemporally continuous disentangled Video representation based upon StyleGAN and Neural-ODEs. Effective traversal of the latent space learned by Generative Adversarial Networks (GANs) has been the basis for recent breakthroughs in image editing. However, the applicability of such advancements to the video domain has been hindered by the difficulty of representing and controlling videos in the latent space of GANs. In particular, videos are composed of content (i.e., appearance) and complex motion components that require a special mechanism to disentangle and control. To achieve this, VidStyleODE encodes the video content in a pre-trained StyleGAN W_+ space and benefits from a latent ODE component to summarize the spatiotemporal dynamics of the input video. Our novel continuous video generation process then combines the two to generate high-quality and temporally consistent videos with varying frame rates. We show that our proposed method enables a variety of applications on real videos: text-guided appearance manipulation, motion manipulation, image animation, and video interpolation and extrapolation. Project website: https://cyberiada.github.io/VidStyleODE

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 51 of its 59 references that are in the map. First posted on arXiv 12 Apr 2023.

Builds on

51 of 59 references.

References · 51

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  5. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  6. 2018Neural Ordinary Differential Equationscited by 3,779
  7. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  8. 2016Improved Techniques for Training GANscited by 2,970
  9. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 2,010
  10. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,234
  11. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 991
  12. 2020Encoding in Style: a StyleGAN Encoder for Image-to-Image Translationcited by 727
38 more
  1. 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 658
  2. 2021Designing an Encoder for StyleGAN Image Manipulationcited by 544
  3. 2017MoCoGAN: Decomposing Motion and Content for Video Generationcited by 524
  4. 2021Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesiscited by 464
  5. 2021Pivotal Tuning for Latent-based Editing of Real Imagescited by 427
  6. 2018Video-to-Video Synthesiscited by 390
  7. 2021GAN Inversion: A Surveycited by 367
  8. 2021StyleGAN-V: A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN2cited by 301
  9. 2022FlowFormer: A Transformer Architecture for Optical Flowcited by 275
  10. 2021Motion Representations for Articulated Animationcited by 255
  11. 2018Learning Blind Video Temporal Consistencycited by 253
  12. 2020Swapping Autoencoder for Deep Image Manipulationcited by 215
  13. 2022Splicing ViT Features for Semantic Appearance Transfercited by 209
  14. 2022Latent Image Animator: Learning to Animate Images via Latent Space Navigationcited by 175
  15. 2022StyleHEAT: One-Shot High-Resolution Editable Talking Face Generation via Pre-trained StyleGANcited by 169
  16. 2021A Good Image Generator Is What You Need for High-Resolution Video Synthesiscited by 163
  17. 2019ManiGAN: Text-Guided Image Manipulationcited by 156
  18. 2015Delving Deeper into Convolutional Networks for Learning Video Representationscited by 148
  19. 2022StyleGAN-Human: A Data-Centric Odyssey of Human Generationcited by 144
  20. 2018Train Sparsely, Generate Densely: Memory-efficient Unsupervised Training of High-resolution Temporal GANcited by 120
  21. 2021Paint by Wordcited by 107
  22. 2015Describing Videos by Exploiting Temporal Structurecited by 88
  23. 2020Semantic Image Manipulation Using Scene Graphscited by 82
  24. 2021HairCLIP: Design Your Hair by Text and Reference Imagecited by 80
  25. 2022Stitch it in Time: GAN-Based Facial Editing of Real Videoscited by 68
  26. 2021A Latent Transformer for Disentangled Face Editing in Images and Videoscited by 67
  27. 2022Third Time's the Charm? Image and Video Editing with StyleGAN3cited by 54
  28. 2020CaSPR: Learning Canonical Spatiotemporal Point Cloud Representationscited by 51
  29. 2021StyleVideoGAN: A Temporal Generative Model using a Pretrained StyleGANcited by 47
  30. 2022InsetGAN for Full-Body Image Generationcited by 45
  31. 2020Vid-ODE: Continuous-Time Video Generation with Neural Ordinary Differential Equationcited by 40
  32. 2019ODE2VAE: Deep generative second order ODEs with Bayesian neural networkscited by 36
  33. 2021Sound-Guided Semantic Image Manipulationcited by 35
  34. 2022Learning Visual Styles from Audio-Visual Associationscited by 22
  35. 2021InMoDeGAN: Interpretable Motion Decomposition Generative Adversarial Network for Video Generationcited by 18
  36. 2022StyleFaceV: Face Video Generation via Decomposing and Recomposing Pretrained StyleGAN3cited by 11
  37. 2021Simple Video Generation using Neural ODEscited by 9
  38. 2022Video2StyleGAN: Disentangling Local and Global Variations in a Videocited by 8

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 0; 2025: 0; 2026: 1.

3 papers.

  1. 2023ElasticDiffusion: Training-free Arbitrary Size Image Generation through Global-Local Content Separationcited by 37
  2. 2023Variational Inference for SDEs Driven by Fractional Noisecited by 9
  3. 2026Beyond Gaussian Bottlenecks: Topologically Aligned Encoding of Vision-Transformer Feature Spacescited by 1