mlsanityData to 7 Oct 2026

Video Frame Synthesis using Deep Voxel Flow

arXiv 1702.02463 ↗First posted 8 Feb 2017cs.CV cs.GR cs.LGcited by 209

Not on the map. Why?

Abstract (arXiv)

We address the problem of synthesizing new video frames in an existing video, either in-between existing frames (interpolation), or subsequent to them (extrapolation). This problem is challenging because video appearance and motion can be highly complex. Traditional optical-flow-based solutions often fail where flow estimation is challenging, while newer neural-network-based methods that hallucinate pixel values directly often produce blurry results. We combine the advantages of these two methods by training a deep network that learns to synthesize video frames by flowing pixel values from existing ones, which we call deep voxel flow. Our method requires no human supervision, and any video can be used as training data by dropping, and then learning to predict, existing frames. The technique is efficient, and can be applied at any video resolution. We demonstrate that our method produces results that both quantitatively and qualitatively improve upon the state-of-the-art.

Key facts. Cited by 209 papers in the map (data through 7 Oct 2026). First posted on arXiv 8 Feb 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 1; 2020: 0; 2021: 50; 2022: 74; 2023: 33; 2024: 24; 2025: 16; 2026: 11.

209 papers.

Sort:

By area:

  1. 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Modelscited by 401
  2. 2022Diffusion Probabilistic Modeling for Video Generationcited by 237
  3. 2022VRT: A Video Restoration Transformercited by 219
  4. 2022SimVP: Simpler yet Better Video Predictioncited by 181
  5. 2023AMT: All-Pairs Multi-Field Transforms for Efficient Frame Interpolationcited by 177
  6. 2021TimeLens: Event-based Video Frame Interpolationcited by 150
  7. 2022FILM: Frame Interpolation for Large Motioncited by 149
  8. 2021XVFI: eXtreme Video Frame Interpolationcited by 123
  9. 2022IFRNet: Intermediate Feature Refine Network for Efficient Frame Interpolationcited by 107
  10. 2021Asymmetric Bilateral Motion Estimation for Video Frame Interpolationcited by 101
  11. 2023Extracting Motion and Appearance via Inter-Frame Attention for Efficient Video Frame Interpolationcited by 97
  12. 2021LAPAR: Linearly-Assembled Pixel-Adaptive Regression Network for Single Image Super-Resolution and Beyondcited by 88
38 more
  1. 2023LDMVFI: Video Frame Interpolation with Latent Diffusion Modelscited by 88
  2. 2021FitVid: Overfitting in Pixel-Level Video Predictioncited by 86
  3. 2022Temporal Attention Unit: Towards Efficient Spatiotemporal Predictive Learningcited by 84
  4. 2021Greedy Hierarchical Variational Autoencoders for Large-Scale Video Predictioncited by 75
  5. 2021Deep Animation Video Interpolation in the Wildcited by 65
  6. 2022High-resolution Face Swapping via Latent Semantics Disentanglementcited by 57
  7. 2021CDFI: Compression-Driven Network Design for Frame Interpolationcited by 55
  8. 2021MotionRNN: A Flexible Model for Video Prediction with Spacetime-Varying Motionscited by 54
  9. 2023A Dynamic Multi-Scale Voxel Flow Network for Video Predictioncited by 53
  10. 2023OpenSTL: A Comprehensive Benchmark of Spatio-Temporal Predictive Learningcited by 53
  11. 2022RSTT: Real-time Spatial Temporal Transformer for Space-Time Video Super-Resolutioncited by 44
  12. 2023PastNet: Introducing Physical Inductive Biases for Spatio-temporal Video Predictioncited by 44
  13. 2022Many-to-many Splatting for Efficient Video Frame Interpolationcited by 42
  14. 2022A Unified Pyramid Recurrent Network for Video Frame Interpolationcited by 39
  15. 2021Understanding Object Dynamics for Interactive Image-to-Video Synthesiscited by 38
  16. 2022TimeReplayer: Unlocking the Potential of Event Cameras for Video Interpolationcited by 37
  17. 2021ST-MFNet: A Spatio-Temporal Multi-Flow Network for Frame Interpolationcited by 34
  18. 2022FloLPIPS: A Bespoke Video Quality Metric for Frame Interpoationcited by 34
  19. 2021SLAMP: Stochastic Latent Appearance and Motion Predictioncited by 32
  20. 2024TANGO: Co-Speech Gesture Video Reenactment with Hierarchical Audio Motion Embedding and Diffusion Interpolationcited by 32
  21. 2022Hard-Constrained Deep Learning for Climate Downscalingcited by 29
  22. 2021End-to-End Rate-Distortion Optimized Learned Hierarchical Bi-Directional Video Compressioncited by 28
  23. 2022Optimizing Video Prediction via Video Frame Interpolationcited by 28
  24. 2021Video Frame Interpolation without Temporal Priorscited by 24
  25. 2023Fourier Neural Operators for Arbitrary Resolution Climate Data Downscalingcited by 24
  26. 2023BiFormer: Learning Bilateral Motion Estimation via Bilateral Transformer for 4K Video Frame Interpolationcited by 23
  27. 2024Generative Inbetweening through Frame-wise Conditions-Driven Video Generationcited by 23
  28. 2023Drag-A-Video: Non-rigid Video Editing with Point-based Interactioncited by 22
  29. 2024VFIMamba: Video Frame Interpolation with State Space Modelscited by 22
  30. 2021Learning Semantic-Aware Dynamics for Video Predictioncited by 21
  31. 2024Explorative Inbetweening of Time and Spacecited by 21
  32. 2021SUNet: Symmetric Undistortion Network for Rolling Shutter Correctioncited by 20
  33. 2022Splatting-based Synthesis for Video Frame Interpolationcited by 20
  34. 2022SimVPv2: Towards Simple yet Powerful Spatiotemporal Predictive Learningcited by 20
  35. 2021Learning for Unconstrained Space-Time Video Super-Resolutioncited by 18
  36. 2022Audio-driven Neural Gesture Reenactment with Video Motion Graphscited by 18
  37. 2022Video Interpolation by Event-driven Anisotropic Adjustment of Optical Flowcited by 18
  38. 2021DeMFI: Deep Joint Deblurring and Multi-Frame Interpolation with Flow-Guided Attentive Correlation and Recursive Boostingcited by 17

50 most cited of 209 All 209 (JSON)