mlsanityData to 7 Oct 2026

SimVP: Simpler yet Better Video Prediction

arXiv 2206.05099 ↗First posted 9 Jun 2022cs.CV cs.AIcited by 181

Not on the map. Why?

Abstract (arXiv)

From CNN, RNN, to ViT, we have witnessed remarkable advancements in video prediction, incorporating auxiliary inputs, elaborate neural architectures, and sophisticated training strategies. We admire these progresses but are confused about the necessity: is there a simple method that can perform comparably well? This paper proposes SimVP, a simple video prediction model that is completely built upon CNN and trained by MSE loss in an end-to-end fashion. Without introducing any additional tricks and complicated strategies, we can achieve state-of-the-art performance on five benchmark datasets. Through extended experiments, we demonstrate that SimVP has strong generalization and extensibility on real-world datasets. The significant reduction of training cost makes it easier to scale to complex scenarios. We believe SimVP can serve as a solid baseline to stimulate the further development of video prediction. The code is available at hrefhttps://github.com/gaozhangyang/SimVP-Simpler-yet-Better-Video-PredictionGithub.

Key facts. Cited by 181 papers in the map (data through 7 Oct 2026). Builds on 56 of its 77 references that are in the map. First posted on arXiv 9 Jun 2022.

Builds on

56 of 77 references.

References · 56

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  4. 2017Neural Discrete Representation Learningcited by 4,650
  5. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  6. 2021ViViT: A Video Vision Transformercited by 1,610
  7. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,609
  8. 2015Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcastingcited by 1,428
  9. 2021Video Swin Transformercited by 1,090
  10. 2021Multiscale Vision Transformerscited by 896
  11. 2017MoCoGAN: Decomposing Motion and Content for Video Generationcited by 523
  12. 2015Unsupervised Learning of Video Representations using LSTMscited by 434
38 more
  1. 2016Generating Videos with Scene Dynamicscited by 376
  2. 2019Axial Attention in Multidimensional Transformerscited by 350
  3. 2016Deep Spatio-Temporal Residual Networks for Citywide Crowd Flows Predictioncited by 330
  4. 2016Dynamic Filter Networkscited by 318
  5. 2015Deep multi-scale video prediction beyond mean square errorcited by 297
  6. 2021Video Transformer Networkcited by 271
  7. 2016Unsupervised Learning for Physical Interaction through Video Predictioncited by 256
  8. 2018Stochastic Video Generation with a Learned Priorcited by 244
  9. 2017Stochastic Variational Video Predictioncited by 234
  10. 2017Video Frame Synthesis using Deep Voxel Flowcited by 209
  11. 2016Deep Predictive Coding Networks for Video Prediction and Unsupervised Learningcited by 203
  12. 2016Temporal Generative Adversarial Nets with Singular Value Clippingcited by 198
  13. 2020Disentangling Physical Dynamics from Unknown Factors for Unsupervised Video Predictioncited by 192
  14. 2019Scaling Autoregressive Video Modelscited by 189
  15. 2016A guide to convolution arithmetic for deep learningcited by 182
  16. 2021PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learningcited by 181
  17. 2018Stochastic Adversarial Video Predictioncited by 173
  18. 2017Decomposing Motion and Content for Natural Video Sequence Predictioncited by 168
  19. 2017Unsupervised Learning of Disentangled Representations from Videocited by 138
  20. 2018PredRNN++: Towards A Resolution of the Deep-in-Time Dilemma in Spatiotemporal Predictive Learningcited by 122
  21. 2018Learning to Decompose and Disentangle Representations for Video Predictioncited by 120
  22. 2020A Review on Deep Learning Techniques for Video Predictioncited by 114
  23. 2016Video Pixel Networkscited by 108
  24. 2020Latent Video Transformercited by 106
  25. 2019Improved Conditional VRNNs for Video Predictioncited by 104
  26. 2014Video (language) modeling: a baseline for generative models of natural videoscited by 92
  27. 2018Memory In Memory: A Predictive Neural Network for Learning Higher-Order Non-Stationarity from Spatiotemporal Dynamicscited by 91
  28. 2017Learning to Generate Long-term Future via Hierarchical Predictioncited by 88
  29. 2017Dual Motion GAN for Future-Flow Embedded Video Predictioncited by 69
  30. 2020Transformation-based Adversarial Video Prediction on Large-Scale Datacited by 66
  31. 2020Exploring Spatial-Temporal Multi-Frequency Analysis for High-Fidelity and Temporal-Consistency Video Predictioncited by 52
  32. 2018Hierarchical Long-term Video Prediction without Supervisioncited by 50
  33. 2018SDCNet: Video Prediction Using Spatially-Displaced Convolutioncited by 50
  34. 2020Probabilistic Future Prediction for Video Scene Understandingcited by 45
  35. 2017Folded Recurrent Neural Networks for Future Video Predictioncited by 44
  36. 2018Disentangling Propagation and Generation for Video Predictioncited by 44
  37. 2018Towards High Resolution Video Generation with Progressive Growing of Sliced Wasserstein GANscited by 36
  38. 2015Spatio-temporal video autoencoder with differentiable memorycited by 32

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 8; 2023: 29; 2024: 52; 2025: 42; 2026: 50.

181 papers.

Sort:

By area:

  1. 2024UniST: A Prompt-Empowered Universal Model for Urban Spatio-Temporal Predictioncited by 86
  2. 2022Temporal Attention Unit: Towards Efficient Spatiotemporal Predictive Learningcited by 84
  3. 2022PiFold: Toward effective and efficient protein inverse foldingcited by 82
  4. 2022MogaNet: Multi-order Gated Aggregation Networkcited by 59
  5. 2023DiffCast: A Unified Framework via Residual Diffusion for Precipitation Nowcastingcited by 57
  6. 2023OpenSTL: A Comprehensive Benchmark of Spatio-Temporal Predictive Learningcited by 53
  7. 2025Frequency-Aligned Knowledge Distillation for Lightweight Spatiotemporal Forecastingcited by 53
  8. 2024CasCast: Skillful High-resolution Precipitation Nowcasting via Cascaded Modellingcited by 52
  9. 2023Seer: Language Instructed Video Prediction with Latent Diffusion Modelscited by 51
  10. 2023CVT-SLR: Contrastive Visual-Textual Transformation for Sign Language Recognition with Variational Alignmentcited by 44
  11. 2023PastNet: Introducing Physical Inductive Biases for Spatio-temporal Video Predictioncited by 44
  12. 2024DINO-Foresight: Looking into the Future with DINOcited by 44
38 more
  1. 2024Towards Generalist Robot Learning from Internet Video: A Surveycited by 40
  2. 2024VMRNN: Integrating Vision Mamba and LSTM for Efficient and Accurate Spatiotemporal Forecastingcited by 35
  3. 2025OneForecast: A Universal Framework for Global and Regional Weather Forecastingcited by 30
  4. 2025AutoReproduce: Automatic AI Experiment Reproduction with Paper Lineagecited by 29
  5. 2024General surgery vision transformer: A video pre-trained foundation model for general surgerycited by 25
  6. 2024Generalizing Weather Forecast to Fine-grained Temporal Scales via Physics-AI Hybrid Modelingcited by 23
  7. 2022SimVPv2: Towards Simple yet Powerful Spatiotemporal Predictive Learningcited by 20
  8. 2024AID: Adapting Image2Video Diffusion Models for Instruction-guided Video Predictioncited by 20
  9. 2023Object-Centric Video Prediction via Decoupling of Object Dynamics and Interactionscited by 19
  10. 2025Sparse identification of nonlinear dynamics and Koopman operators with Shallow Recurrent Decoder Networkscited by 19
  11. 2023MMVP: Motion-Matrix-based Video Predictioncited by 18
  12. 2022MIMO Is All You Need : A Strong Multi-In-Multi-Out Baseline for Video Predictioncited by 17
  13. 2023Multi-modal learning for geospatial vegetation forecastingcited by 16
  14. 2023Earthfarseer: Versatile Spatio-Temporal Dynamical Systems Modeling in One Modelcited by 16
  15. 2024Fourier Amplitude and Correlation Loss: Beyond Using L2 Loss for Skillful Precipitation Nowcastingcited by 16
  16. 2023Implicit Stacked Autoregressive Model for Video Predictioncited by 14
  17. 2024Video Representation Learning with Joint-Embedding Predictive Architecturescited by 14
  18. 2024Retrieval Meets Reasoning: Even High-school Textbook Knowledge Benefits Multimodal Reasoningcited by 13
  19. 2023Knowledge-Design: Pushing the Limit of Protein Design via Knowledge Refinementcited by 11
  20. 2023Enhancing Human-like Multi-Modal Reasoning: A New Challenging Dataset and Comprehensive Frameworkcited by 11
  21. 2024GenRec: Unifying Video Generation and Recognition with Diffusion Modelscited by 11
  22. 2024E-Motion: Future Motion Simulation via Event Sequence Diffusioncited by 11
  23. 2024A Graph is Worth K Words: Euclideanizing Graph using Pure Transformercited by 10
  24. 2024PostCast: Generalizable Postprocessing for Precipitation Nowcasting via Unsupervised Blurriness Modelingcited by 10
  25. 2024Diffusion Transformers as Open-World Spatiotemporal Foundation Modelscited by 10
  26. 2023Deterministic Guidance Diffusion Model for Probabilistic Weather Forecastingcited by 9
  27. 2024Modeling Spatio-temporal Dynamical Systems with Neural Discrete Learning and Levels-of-Expertscited by 9
  28. 2024Switch EMA: A Free Lunch for Better Flatness and Sharpnesscited by 9
  29. 2023Cross-Gate MLP with Protein Complex Invariant Embedding is A One-Shot Antibody Designercited by 8
  30. 2025RadarQA: Multi-modal Quality Analysis of Weather Radar Forecastscited by 8
  31. 2023PrefixMol: Target- and Chemistry-aware Molecule Design via Prefix Embeddingcited by 7
  32. 2023Triplet Attention Transformer for Spatiotemporal Predictive Learningcited by 7
  33. 2024A Survey on Future Frame Synthesis: Bridging Deterministic and Generative Approachescited by 7
  34. 2024DynST: Dynamic Sparse Training for Resource-Constrained Spatio-Temporal Forecastingcited by 7
  35. 2024Self-supervised Multi-future Occupancy Forecasting for Autonomous Drivingcited by 7
  36. 2024UniFlow: A Foundation Model for Unified Urban Spatio-Temporal Flow Predictioncited by 7
  37. 2025Advancing Semantic Future Prediction through Multimodal Visual Sequence Transformerscited by 7
  38. 2025EgoControl: Controllable Egocentric Video Generation via 3D Full-Body Posescited by 7

50 most cited of 181 All 181 (JSON)