mlsanityData to 7 Oct 2026

A Dynamic Multi-Scale Voxel Flow Network for Video Prediction

arXiv 2303.09875 ↗First posted 17 Mar 2023cs.CVcited by 53

Not on the map. Why?

Abstract (arXiv)

The performance of video prediction has been greatly boosted by advanced deep neural networks. However, most of the current methods suffer from large model sizes and require extra inputs, e.g., semantic/depth maps, for promising performance. For efficiency consideration, in this paper, we propose a Dynamic Multi-scale Voxel Flow Network (DMVFN) to achieve better video prediction performance at lower computational costs with only RGB images, than previous methods. The core of our DMVFN is a differentiable routing module that can effectively perceive the motion scales of video frames. Once trained, our DMVFN selects adaptive sub-networks for different inputs at the inference stage. Experiments on several benchmarks demonstrate that our DMVFN is an order of magnitude faster than Deep Voxel Flow and surpasses the state-of-the-art iterative-based OPT on generated image quality. Our code and demo are available at https://huxiaotaostasy.github.io/DMVFN/.

Key facts. Cited by 53 papers in the map (data through 7 Oct 2026). Builds on 69 of its 81 references that are in the map. First posted on arXiv 17 Mar 2023.

Builds on

69 of 81 references.

References · 69

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  3. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  4. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  5. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  6. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
  7. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  8. 2017Decoupled Weight Decay Regularizationcited by 1,722
  9. 2017Deformable Convolutional Networkscited by 1,481
  10. 2018World Modelscited by 1,444
  11. 2015Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcastingcited by 1,428
  12. 2015Spatial Transformer Networkscited by 1,423
38 more
  1. 2023Mastering Diverse Domains through World Modelscited by 1,193
  2. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,054
  3. 2017PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volumecited by 966
  4. 2020Designing Network Design Spacescited by 954
  5. 2017The 2017 DAVIS Challenge on Video Object Segmentationcited by 942
  6. 2016FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networkscited by 780
  7. 2018Deformable ConvNets v2: More Deformable, Better Resultscited by 666
  8. 2018BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentationcited by 645
  9. 2017Video Enhancement with Task-Oriented Flowcited by 612
  10. 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 591
  11. 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 557
  12. 2016Optical Flow Estimation using a Spatial Pyramid Networkcited by 421
  13. 2016Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activationscited by 413
  14. 2021Dynamic Neural Networks: A Surveycited by 405
  15. 2018Video-to-Video Synthesiscited by 388
  16. 2017On human motion prediction using recurrent neural networkscited by 336
  17. 2017SkipNet: Learning Dynamic Routing in Convolutional Networkscited by 328
  18. 2021GMFlow: Learning Optical Flow via Global Matchingcited by 324
  19. 2017Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolationcited by 321
  20. 2022FlowFormer: A Transformer Architecture for Optical Flowcited by 273
  21. 2017Video Frame Synthesis using Deep Voxel Flowcited by 209
  22. 2016Deep Predictive Coding Networks for Video Prediction and Unsupervised Learningcited by 203
  23. 2016View Synthesis by Appearance Flowcited by 172
  24. 2017Decomposing Motion and Content for Natural Video Sequence Predictioncited by 168
  25. 2017Adaptive Neural Networks for Efficient Inferencecited by 161
  26. 2017Convolutional Networks with Adaptive Inference Graphscited by 147
  27. 2019Quadratic video interpolationcited by 143
  28. 2018AdaFrame: Adaptive Frame Selection for Fast Video Recognitioncited by 134
  29. 2018Improving Semantic Segmentation via Video Propagation and Label Relaxationcited by 134
  30. 2019Pixel-Adaptive Convolutional Neural Networkscited by 126
  31. 2020What Matters in Unsupervised Optical Flowcited by 124
  32. 2021XVFI: eXtreme Video Frame Interpolationcited by 123
  33. 2020A Review on Deep Learning Techniques for Video Predictioncited by 114
  34. 2022IFRNet: Intermediate Feature Refine Network for Efficient Frame Interpolationcited by 107
  35. 2019Improved Conditional VRNNs for Video Predictioncited by 104
  36. 2015End-to-end Learning of Action Detection from Frame Glimpses in Videoscited by 101
  37. 2023Extracting Motion and Appearance via Inter-Frame Attention for Efficient Video Frame Interpolationcited by 97
  38. 2019Dynamic Convolutions: Exploiting Spatial Sparsity for Faster Inferencecited by 91

50 most cited of 69 All 69 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 16; 2024: 24; 2025: 7; 2026: 6.

53 papers.

Sort:

By area:

  1. 2023DynamiCrafter: Animating Open-domain Images with Video Diffusion Priorscited by 463
  2. 2023AMT: All-Pairs Multi-Field Transforms for Efficient Frame Interpolationcited by 177
  3. 2024GenAD: Generalized Predictive Model for Autonomous Drivingcited by 150
  4. 2023Generative Image Dynamicscited by 93
  5. 2023LAMP: Learn A Motion Pattern for Few-Shot-Based Video Generationcited by 59
  6. 2023OpenSTL: A Comprehensive Benchmark of Spatio-Temporal Predictive Learningcited by 53
  7. 2023PastNet: Introducing Physical Inductive Biases for Spatio-temporal Video Predictioncited by 44
  8. 2024VMRNN: Integrating Vision Mamba and LSTM for Efficient and Accurate Spatiotemporal Forecastingcited by 35
  9. 2024VLMPC: Vision-Language Model Predictive Control for Robotic Manipulationcited by 32
  10. 2023MMVP: Motion-Matrix-based Video Predictioncited by 18
  11. 2023Earthfarseer: Versatile Spatio-Temporal Dynamical Systems Modeling in One Modelcited by 16
  12. 2023Event-based Continuous Color Video Decompression from Single Framescited by 14
38 more
  1. 2023Automatic Animation of Hair Blowing in Still Portrait Photoscited by 13
  2. 2023Context-PIPs: Persistent Independent Particles Demands Spatial Context Featurescited by 9
  3. 2023Deterministic Guidance Diffusion Model for Probabilistic Weather Forecastingcited by 9
  4. 2023Scale-Adaptive Feature Aggregation for Efficient Space-Time Video Super-Resolutioncited by 8
  5. 2024Exposing AI-generated Videos: A Benchmark Dataset and a Local-and-Global Temporal Defect Based Detection Methodcited by 8
  6. 2025H-MoRe: Learning Human-centric Motion Representation for Action Analysiscited by 8
  7. 2023Triplet Attention Transformer for Spatiotemporal Predictive Learningcited by 7
  8. 2024A Survey on Future Frame Synthesis: Bridging Deterministic and Generative Approachescited by 7
  9. 2024AFter: Attention-based Fusion Router for RGBT Trackingcited by 7
  10. 2024Using Left and Right Brains Together: Towards Vision and Language Planningcited by 6
  11. 2024Video Prediction Transformers without Recurrence or Convolutioncited by 6
  12. 2024Advection Augmented Convolutional Neural Networkscited by 5
  13. 2024Motion Graph Unleashed: A Novel Approach to Video Predictioncited by 4
  14. 2025Vision-Language Model Predictive Control for Manipulation Planning and Trajectory Generationcited by 4
  15. 2023Does Vector Quantization Fail in Spatio-Temporal Forecasting? Exploring a Differentiable Sparse Soft-Vector Quantization Approachcited by 3
  16. 2024OCK: Unsupervised Dynamic Video Prediction with Object-Centric Kinematicscited by 3
  17. 2024Appearance Blur-driven AutoEncoder and Motion-guided Memory Module for Video Anomaly Detectioncited by 3
  18. 2025One Step Closer: Creating the Future to Boost Monocular Semantic Scene Completioncited by 3
  19. 2024Artificial Intelligence for Biomedical Video Generationcited by 2
  20. 2025Unified Arbitrary-Time Video Frame Interpolation and Predictioncited by 2
  21. 2024Adaptive Runge-Kutta Dynamics for Spatiotemporal Predictioncited by 1
  22. 2024GFFE: G-buffer Free Frame Extrapolation for Low-latency Real-time Renderingcited by 1
  23. 2024Forecasting Future Videos from Novel Views via Disentangled 3D Scene Representationcited by 1
  24. 2024STLight: a Fully Convolutional Approach for Efficient Predictive Learning by Spatio-Temporal joint Processingcited by 1
  25. 2024ARCON: Advancing Auto-Regressive Continuation for Driving Videoscited by 1
  26. 2025Real-time Video Prediction With Fast Video Interpolation Model and Prediction Trainingcited by 1
  27. 2025DESSERT: Diffusion-based Event-driven Single-frame Synthesis via Residual Trainingcited by 1
  28. 2026BEVPredFormer: Spatio-temporal Attention for BEV Instance Prediction in Autonomous Drivingcited by 1
  29. 2023Lightweight Improved Residual Network for Efficient Inverse Tone Mappingcited by 0
  30. 2024Dynamic neural network with memristive CIM and CAM for 2D and 3D visioncited by 0
  31. 2024Task-Aware Dynamic Transformer for Efficient Arbitrary-Scale Image Super-Resolutioncited by 0
  32. 2024Towards Real-Time Generation of Delay-Compensated Video Feeds for Outdoor Mobile Robot Teleoperationcited by 0
  33. 2024CausalVE: Face Video Privacy Encryption via Causal Video Predictioncited by 0
  34. 2024Data-Driven Uncertainty-Aware Forecasting of Sea Ice Conditions in the Gulf of Ob Based on Satellite Radar Imagerycited by 0
  35. 2024Lightweight Stochastic Video Prediction via Hybrid Warpingcited by 0
  36. 2025RAPTOR: Real-Time High-Resolution UAV Video Prediction with Efficient Video Attentioncited by 0
  37. 2026CloDS: Visual-Only Unsupervised Cloth Dynamics Learning in Unknown Conditionscited by 0
  38. 2026PFGNet: A Fully Convolutional Frequency-Guided Peripheral Gating Network for Efficient Spatiotemporal Predictive Learningcited by 0

50 most cited of 53 All 53 (JSON)