mlsanityData to 7 Oct 2026

Flow-Grounded Spatial-Temporal Video Prediction from Still Images

arXiv 1807.09755 ↗First posted 25 Jul 2018cs.CVcited by 54

Not on the map. Why?

Abstract (arXiv)

Existing video prediction methods mainly rely on observing multiple historical frames or focus on predicting the next one-frame. In this work, we study the problem of generating consecutive multiple future frames by observing one single still image only. We formulate the multi-frame prediction task as a multiple time step flow (multi-flow) prediction phase followed by a flow-to-frame synthesis phase. The multi-flow prediction is modeled in a variational probabilistic manner with spatial-temporal relationships learned through 3D convolutions. The flow-to-frame synthesis is modeled as a generative process in order to keep the predicted results lying closer to the manifold shape of real video sequence. Such a two-phase design prevents the model from directly looking at the high-dimensional pixel space of the frame sequence and is demonstrated to be more effective in predicting better and diverse results. Extensive experimental results on videos with different types of motion show that the proposed algorithm performs favorably against existing methods in terms of quality, diversity and human perceptual evaluation.

Key facts. Cited by 54 papers in the map (data through 7 Oct 2026). First posted on arXiv 25 Jul 2018.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 20; 2022: 11; 2023: 10; 2024: 7; 2025: 6.

54 papers.

Sort:

By area:

  1. 2023DynamiCrafter: Animating Open-domain Images with Video Diffusion Priorscited by 463
  2. 2021GAN Inversion: A Surveycited by 366
  3. 2023Conditional Image-to-Video Generation with Latent Flow Diffusion Modelscited by 180
  4. 2022Latent Image Animator: Learning to Animate Images via Latent Space Navigationcited by 172
  5. 2023Diffusion with Forward Models: Solving Stochastic Inverse Problems Without Direct Supervisioncited by 112
  6. 2021Make It Move: Controllable Image-to-Video Generation with Text Descriptionscited by 88
  7. 2024MOFA-Video: Controllable Image Animation via Generative Motion Field Adaptions in Frozen Image-to-Video Diffusion Modelcited by 87
  8. 2021Predicting Video with VQVAEcited by 61
  9. 2022Look Outside the Room: Synthesizing A Consistent Long-Term 3D Scene Video from A Single Imagecited by 55
  10. 2021Controllable Animation of Fluid Elements in Still Imagescited by 44
  11. 2024Moonshot: Towards Controllable Video Generation and Editing with Multimodal Conditionscited by 42
  12. 2023LEO: Generative Latent Image Animator for Human Video Synthesiscited by 40
38 more
  1. 2021Understanding Object Dynamics for Interactive Image-to-Video Synthesiscited by 38
  2. 2025Free4D: Tuning-free 4D Scene Generation with Spatial-Temporal Consistencycited by 34
  3. 2021SLAMP: Stochastic Latent Appearance and Motion Predictioncited by 32
  4. 2024TRIP: Temporal Residual Learning with Image Noise Prior for Image-to-Video Diffusion Modelscited by 30
  5. 20233D Cinemagraphy from a Single Imagecited by 23
  6. 2021Learning Semantic-Aware Dynamics for Video Predictioncited by 21
  7. 2023LaMD: Latent Motion Diffusion for Image-Conditional Video Generationcited by 21
  8. 2023Make-It-4D: Synthesizing a Consistent Long-Term Dynamic Scene Video from a Single Imagecited by 19
  9. 2021InMoDeGAN: Interpretable Motion Decomposition Generative Adversarial Network for Video Generationcited by 18
  10. 2024Optical-Flow Guided Prompt Optimization for Coherent Video Generationcited by 17
  11. 2023Blowing in the Wind: CycleNet for Human Cinemagraphs from Still Imagescited by 14
  12. 2024Dance Any Beat: Blending Beats with Visuals in Dance Video Generationcited by 14
  13. 2022MSPred: Video Prediction at Multiple Spatio-Temporal Scales with Hierarchical Recurrent Networkscited by 12
  14. 2021Click to Move: Controlling Video Generation with Sparse Motioncited by 11
  15. 2021MaCLR: Motion-aware Contrastive Learning of Representations for Videoscited by 10
  16. 2021Nowcasting-Nets: Deep Neural Network Structures for Precipitation Nowcasting Using IMERGcited by 10
  17. 2021Conditional Temporal Variational AutoEncoder for Action Video Predictioncited by 9
  18. 2025What Happens Next? Anticipating Future Motion by Generating Point Trajectoriescited by 9
  19. 2021Learning Fine-Grained Motion Embedding for Landscape Animationcited by 7
  20. 2021Multi-Object Tracking with Hallucinated and Unlabeled Videoscited by 6
  21. 2022WALDO: Future Video Synthesis using Object Layer Decomposition and Parametric Flow Predictioncited by 5
  22. 2024Tuning-Free Noise Rectification for High Fidelity Image-to-Video Generationcited by 5
  23. 2022Text-driven Video Predictioncited by 4
  24. 2022Visual Semantic Parsing: From Images to Abstract Meaning Representationcited by 4
  25. 2023Versatile Face Animator: Driving Arbitrary 3D Facial Avatar in RGBD Spacecited by 4
  26. 2024AnaMoDiff: 2D Analogical Motion Diffusion via Disentangled Denoisingcited by 4
  27. 2025ReVision: Refining Video Diffusion with Explicit 3D Motion Modelingcited by 4
  28. 2025LIA-X: Interpretable Latent Portrait Animatorcited by 4
  29. 2021Dense Semantic Forecasting in Video by Joint Regression of Features and Feature Motioncited by 3
  30. 2023GenDeF: Learning Generative Deformation Field for Video Generationcited by 3
  31. 2021TaylorSwiftNet: Taylor Driven Temporal Modeling for Swift Future Frame Predictioncited by 2
  32. 2022Predictive Coding Based Multiscale Network with Encoder-Decoder LSTM for Video Predictioncited by 2
  33. 2021Dual-MTGAN: Stochastic and Deterministic Motion Transfer for Image-to-Video Synthesiscited by 1
  34. 2021Continuous-Time Video Generation via Learning Motion Dynamics with Neural ODEcited by 1
  35. 2022V3GAN: Decomposing Background, Foreground and Motion for Video Generationcited by 1
  36. 2021Spatial-temporal V-Net for automatic segmentation and quantification of right ventricles in gated myocardial perfusion SPECT imagescited by 0
  37. 2021Wide and Narrow: Video Prediction from Context and Motioncited by 0
  38. 2022Unified Simulation, Perception, and Generation of Human Behaviorcited by 0

50 most cited of 54 All 54 (JSON)