mlsanityData to 7 Oct 2026

Future Video Synthesis with Object Motion Prediction

arXiv 2004.00542 ↗First posted 1 Apr 2020cs.CVcited by 57

Not on the map. Why?

Abstract (arXiv)

We present an approach to predict future video frames given a sequence of continuous video frames in the past. Instead of synthesizing images directly, our approach is designed to understand the complex scene dynamics by decoupling the background scene and moving objects. The appearance of the scene components in the future is predicted by non-rigid deformation of the background and affine transformation of moving objects. The anticipated appearances are combined to create a reasonable video in the future. With this procedure, our method exhibits much less tearing or distortion artifact compared to other approaches. Experimental results on the Cityscapes and KITTI datasets show that our model outperforms the state-of-the-art in terms of visual quality and accuracy.

Key facts. Cited by 57 papers in the map (data through 7 Oct 2026). First posted on arXiv 1 Apr 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 20; 2022: 16; 2023: 9; 2024: 11; 2025: 0; 2026: 1.

57 papers.

Sort:

By area:

  1. 2023DragNUWA: Fine-grained Control in Video Generation by Integrating Text, Image, and Trajectorycited by 260
  2. 2021PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learningcited by 181
  3. 2021Make It Move: Controllable Image-to-Video Generation with Text Descriptionscited by 88
  4. 2022Touch and Go: Learning from Human-Collected Vision and Touchcited by 84
  5. 2021CCVS: Context-aware Controllable Video Synthesiscited by 68
  6. 2021Video Prediction Recalling Long-term Motion Context via Memory Alignment Learningcited by 62
  7. 2023Dense Optical Tracking: Connecting the Dotscited by 58
  8. 2023A Dynamic Multi-Scale Voxel Flow Network for Video Predictioncited by 53
  9. 2021Stochastic Image-to-Video Synthesis using cINNscited by 49
  10. 2021Controllable Animation of Fluid Elements in Still Imagescited by 44
  11. 2023PastNet: Introducing Physical Inductive Biases for Spatio-temporal Video Predictioncited by 44
  12. 2021iPOKE: Poking a Still Image for Controlled Stochastic Video Synthesiscited by 42
38 more
  1. 2021Understanding Object Dynamics for Interactive Image-to-Video Synthesiscited by 38
  2. 2022Optimizing Video Prediction via Video Frame Interpolationcited by 28
  3. 2022Is an Object-Centric Video Representation Beneficial for Transfer?cited by 24
  4. 2021Learning Semantic-Aware Dynamics for Video Predictioncited by 21
  5. 2022VPTR: Efficient Transformers for Video Predictioncited by 17
  6. 2021Comparing Correspondences: Video Prediction with Correspondence-wise Lossescited by 16
  7. 2023Earthfarseer: Versatile Spatio-Temporal Dynamical Systems Modeling in One Modelcited by 16
  8. 2021Action2video: Generating Videos of Human 3D Actionscited by 15
  9. 2022Video Prediction by Efficient Transformerscited by 12
  10. 2023MOSO: Decomposing MOtion, Scene and Object for Video Predictioncited by 12
  11. 2021Click to Move: Controlling Video Generation with Sparse Motioncited by 11
  12. 2021A Hierarchical Variational Neural Uncertainty Model for Stochastic Video Predictioncited by 10
  13. 2024VISAGE: Video Synthesis using Action Graphs for Surgerycited by 9
  14. 2023Event-Free Moving Object Segmentation from Moving Ego Vehiclecited by 8
  15. 2021Panoptic Segmentation Forecastingcited by 7
  16. 2022A unified model for continuous conditional video predictioncited by 7
  17. 2024A Survey on Future Frame Synthesis: Bridging Deterministic and Generative Approachescited by 7
  18. 2024MFTIQ: Multi-Flow Tracker with Independent Matching Quality Estimationcited by 7
  19. 2021Self-Supervision by Prediction for Object Discovery in Videoscited by 6
  20. 2021Unsupervised Video Prediction from a Single Frame by Estimating 3D Dynamic Scene Structurecited by 6
  21. 2021Review of Video Predictive Understanding: Early Action Recognition and Future Action Predictioncited by 5
  22. 2022WALDO: Future Video Synthesis using Object Layer Decomposition and Parametric Flow Predictioncited by 5
  23. 2024Motion Graph Unleashed: A Novel Approach to Video Predictioncited by 4
  24. 2022Joint Forecasting of Panoptic Segmentations with Difference Attentioncited by 3
  25. 2022Temporal View Synthesis of Dynamic Scenes through 3D Object Motion Estimation with Multi-Plane Imagescited by 3
  26. 2023Stable and Consistent Prediction of 3D Characteristic Orientation via Invariant Residual Learningcited by 3
  27. 2026Space-Time Forecasting of Dynamic Scenes with Motion-aware Gaussian Groupingcited by 3
  28. 2021Video Anomaly Detection By The Duality Of Normality-Granted Optical Flowcited by 2
  29. 2021Putting 3D Spatially Sparse Networks on a Dietcited by 2
  30. 2022Pyramidal Predictive Network: A Model for Visual-frame Prediction Based on Predictive Coding Theorycited by 2
  31. 2022Predictive Coding Based Multiscale Network with Encoder-Decoder LSTM for Video Predictioncited by 2
  32. 2024TAFormer: A Unified Target-Aware Transformer for Video and Motion Joint Prediction in Aerial Scenescited by 2
  33. 2024Artificial Intelligence for Biomedical Video Generationcited by 2
  34. 2022Wildfire Forecasting with Satellite Images and Deep Generative Modelcited by 1
  35. 2024Learning Physical Dynamics for Object-centric Visual Predictioncited by 1
  36. 2024Forecasting Future Videos from Novel Views via Disentangled 3D Scene Representationcited by 1
  37. 2024Panoptic-Depth Forecastingcited by 1
  38. 2021From Single to Multiple: Leveraging Multi-level Prediction Spaces for Video Forecastingcited by 0

50 most cited of 57 All 57 (JSON)