mlsanityData to 7 Oct 2026

Stochastic Image-to-Video Synthesis using cINNs

arXiv 2105.04551 ↗First posted 10 May 2021cs.CVcited by 49

Not on the map. Why?

Abstract (arXiv)

Video understanding calls for a model to learn the characteristic interplay between static scene content and its dynamics: Given an image, the model must be able to predict a future progression of the portrayed scene and, conversely, a video should be explained in terms of its static image content and all the remaining characteristics not present in the initial frame. This naturally suggests a bijective mapping between the video domain and the static content as well as residual information. In contrast to common stochastic image-to-video synthesis, such a model does not merely generate arbitrary videos progressing the initial image. Given this image, it rather provides a one-to-one mapping between the residual vectors and the video with stochastic outcomes when sampling. The approach is naturally implemented using a conditional invertible neural network (cINN) that can explain videos by independently modelling static and other video characteristics, thus laying the basis for controlled video synthesis. Experiments on four diverse video datasets demonstrate the effectiveness of our approach in terms of both the quality and diversity of the synthesized results. Our project page is available at https://bit.ly/3t66bnU.

Key facts. Cited by 49 papers in the map (data through 7 Oct 2026). Builds on 79 of its 98 references that are in the map. First posted on arXiv 10 May 2021.

Builds on

79 of 98 references.

References · 79

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  6. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  7. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  8. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  9. 2013Auto-Encoding Variational Bayescited by 3,980
  10. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  11. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,678
  12. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
38 more
  1. 2017Improved Training of Wasserstein GANscited by 2,223
  2. 2015Variational Inference with Normalizing Flowscited by 2,161
  3. 2016Density estimation using Real NVPcited by 2,029
  4. 2017The Kinetics Human Action Video Datasetcited by 1,894
  5. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  6. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,539
  7. 2018Group Normalizationcited by 1,514
  8. 2019Normalizing Flows for Probabilistic Modeling and Inferencecited by 1,315
  9. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,295
  10. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
  11. 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,176
  12. 2014NICE: Non-linear Independent Components Estimationcited by 1,052
  13. 2016FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networkscited by 780
  14. 2020First Order Motion Model for Image Animationcited by 698
  15. 2018Which Training Methods for GANs do actually Converge?cited by 672
  16. 2017MoCoGAN: Decomposing Motion and Content for Video Generationcited by 523
  17. 2015A Neural Algorithm of Artistic Stylecited by 480
  18. 2015Autoencoding beyond pixels using a learned similarity metriccited by 407
  19. 2018Video-to-Video Synthesiscited by 388
  20. 2018Understanding disentangling in β-VAEcited by 332
  21. 2017Toward Multimodal Image-to-Image Translationcited by 295
  22. 2016Deep Visual Foresight for Planning Robot Motioncited by 276
  23. 2020Softmax Splatting for Video Frame Interpolationcited by 276
  24. 2018Stochastic Video Generation with a Learned Priorcited by 244
  25. 2018Analyzing Inverse Problems with Invertible Neural Networkscited by 239
  26. 2017Stochastic Variational Video Predictioncited by 234
  27. 2017Geometric GANcited by 232
  28. 2019Depth-Aware Video Frame Interpolationcited by 232
  29. 2016Generating Images with Perceptual Similarity Metrics based on Deep Networkscited by 221
  30. 2020SRFlow: Learning the Super-Resolution Space with Normalizing Flowcited by 209
  31. 2018Animating Arbitrary Objects via Deep Motion Transfercited by 206
  32. 2019Scaling Autoregressive Video Modelscited by 189
  33. 2016Variational Lossy Autoencodercited by 175
  34. 2018Stochastic Adversarial Video Predictioncited by 173
  35. 2017Decomposing Motion and Content for Natural Video Sequence Predictioncited by 168
  36. 2019Liquid Warping GAN: A Unified Framework for Human Motion Imitation, Appearance Transfer and Novel View Synthesiscited by 161
  37. 2018Context-aware Synthesis for Video Frame Interpolationcited by 153
  38. 2019Guided Image Generation with Conditional Invertible Neural Networkscited by 147

50 most cited of 79 All 79 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 7; 2022: 9; 2023: 15; 2024: 10; 2025: 6; 2026: 2.

49 papers.

Sort:

By area:

  1. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  2. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  3. 2023NExT-GPT: Any-to-Any Multimodal LLMcited by 610
  4. 2023DynamiCrafter: Animating Open-domain Images with Video Diffusion Priorscited by 463
  5. 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Drivingcited by 309
  6. 2022Diffusion Probabilistic Modeling for Video Generationcited by 237
  7. 2023Conditional Image-to-Video Generation with Latent Flow Diffusion Modelscited by 180
  8. 2021ImageBART: Bidirectional Context with Multinomial Diffusion for Autoregressive Image Synthesiscited by 122
  9. 2022MaskViT: Masked Visual Pre-Training for Video Predictioncited by 112
  10. 2023Generative Image Dynamicscited by 93
  11. 2021Make It Move: Controllable Image-to-Video Generation with Text Descriptionscited by 88
  12. 2024MOFA-Video: Controllable Image Animation via Generative Motion Field Adaptions in Frozen Image-to-Video Diffusion Modelcited by 87
37 more
  1. 2021FitVid: Overfitting in Pixel-Level Video Predictioncited by 86
  2. 2024Magic-Me: Identity-Specific Video Customized Diffusioncited by 58
  3. 2024Fr'echet Video Motion Distance: A Metric for Evaluating Motion Consistency in Videoscited by 54
  4. 2024Follow-Your-Click: Open-domain Regional Image Animation via Short Promptscited by 52
  5. 2021Controllable Animation of Fluid Elements in Still Imagescited by 44
  6. 2021iPOKE: Poking a Still Image for Controlled Stochastic Video Synthesiscited by 42
  7. 2021Understanding Object Dynamics for Interactive Image-to-Video Synthesiscited by 38
  8. 2022Show Me What and Tell Me How: Video Synthesis via Multimodal Conditioningcited by 37
  9. 2022Exploring Optical-Flow-Guided Motion and Detection-Based Appearance for Temporal Sentence Groundingcited by 35
  10. 2022Non-isotropy Regularization for Proxy-based Deep Metric Learningcited by 33
  11. 2024TRIP: Temporal Residual Learning with Image Noise Prior for Image-to-Video Diffusion Modelscited by 30
  12. 2023AI-Generated Content (AIGC) for Various Data Modalities: A Surveycited by 25
  13. 2024TI2V-Zero: Zero-Shot Image Conditioning for Text-to-Video Diffusion Modelscited by 25
  14. 2023LaMD: Latent Motion Diffusion for Image-Conditional Video Generationcited by 21
  15. 2025X-Dyna: Expressive Dynamic Human Image Animationcited by 18
  16. 2022SCVRL: Shuffled Contrastive Video Representation Learningcited by 14
  17. 2023Blowing in the Wind: CycleNet for Human Cinemagraphs from Still Imagescited by 14
  18. 2024Dance Any Beat: Blending Beats with Visuals in Dance Video Generationcited by 14
  19. 2024Cinemo: Consistent and Controllable Image Animation with Motion Diffusion Modelscited by 14
  20. 2025Beyond the Frame: Generating 360 Panoramic Videos from Perspective Videoscited by 14
  21. 2023Automatic Animation of Hair Blowing in Still Portrait Photoscited by 13
  22. 2025Ultrasound Image-to-Video Synthesis via Latent Dynamic Diffusion Modelscited by 12
  23. 2022Language-Guided Face Animation by Recurrent StyleGAN-based Generatorcited by 9
  24. 2021A Compact Neural Network-based Algorithm for Robust Image Watermarkingcited by 6
  25. 2023THInImg: Cross-modal Steganography for Presenting Talking Heads in Imagescited by 6
  26. 2025Other Vehicle Trajectories Are Also Needed: A Driving World Model Unifies Ego-Other Vehicle Trajectories in Video Latent Spacecited by 5
  27. 2022Text-driven Video Predictioncited by 4
  28. 2024Rethinking Human Evaluation Protocol for Text-to-Video Models: Enhancing Reliability,Reproducibility, and Practicalitycited by 4
  29. 2024Stereo-Talker: Audio-driven 3D Human Synthesis with Prior-Guided Mixture-of-Expertscited by 4
  30. 2023GenDeF: Learning Generative Deformation Field for Video Generationcited by 3
  31. 2025Consistent and Controllable Image Animation with Motion Linear Diffusion Transformerscited by 3
  32. 2026DreamLoop: Controllable Cinemagraph Generation from a Single Photographcited by 2
  33. 2026Envisioning the Future, One Step at a Timecited by 2
  34. 2023FashionFlow: Leveraging Diffusion Models for Dynamic Fashion Video Synthesis from Static Imagerycited by 1
  35. 2023Infusion: internal diffusion for inpainting of dynamic textures and complex motioncited by 1
  36. 2022Deep Boosting Robustness of DNN-based Image Watermarking via DBMarkcited by 0
  37. 2025Unified Text-Image-to-Video Generation: A Training-Free Approach to Flexible Visual Conditioningcited by 0