mlsanityData to 7 Oct 2026

Softmax Splatting for Video Frame Interpolation

arXiv 2003.05534 ↗First posted 11 Mar 2020cs.CVcited by 276

Not on the map. Why?

Cited by 27 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Differentiable image sampling in the form of backward warping has seen broad adoption in tasks like depth estimation and optical flow prediction. In contrast, how to perform forward warping has seen less attention, partly due to additional challenges such as resolving the conflict of mapping multiple pixels to the same target location in a differentiable way. We propose softmax splatting to address this paradigm shift and show its effectiveness on the application of frame interpolation. Specifically, given two input frames, we forward-warp the frames and their feature pyramid representations based on an optical flow estimate using softmax splatting. In doing so, the softmax splatting seamlessly handles cases where multiple source pixels map to the same target location. We then use a synthesis network to predict the interpolation result from the warped representations. Our softmax splatting allows us to not only interpolate frames at an arbitrary time but also to fine tune the feature pyramid and the optical flow. We show that our synthesis approach, empowered by softmax splatting, achieves new state-of-the-art results for video frame interpolation.

Key facts. Cited by 276 papers in the map (data through 7 Oct 2026). First posted on arXiv 11 Mar 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 41; 2022: 60; 2023: 45; 2024: 60; 2025: 41; 2026: 29.

276 papers.

Sort:

By area:

  1. 2021BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignmentcited by 340
  2. 2021Deep Contextual Video Compressioncited by 216
  3. 2024Motion-I2V: Consistent and Controllable Image-to-Video Generation with Explicit Motion Modelingcited by 209
  4. 2024SpatialTracker: Tracking Any 2D Pixels in 3D Spacecited by 178
  5. 2023AMT: All-Pairs Multi-Field Transforms for Efficient Frame Interpolationcited by 177
  6. 2023SceneScape: Text-Driven Consistent Scene Generationcited by 160
  7. 2021TimeLens: Event-based Video Frame Interpolationcited by 150
  8. 2021E-RAFT: Dense Optical Flow from Event Camerascited by 149
  9. 2022FILM: Frame Interpolation for Large Motioncited by 149
  10. 2021XVFI: eXtreme Video Frame Interpolationcited by 123
  11. 2021Video Frame Interpolation Transformercited by 122
  12. 2023Diffusion with Forward Models: Solving Stochastic Inverse Problems Without Direct Supervisioncited by 112
38 more
  1. 2022Time Lens++: Event-based Frame Interpolation with Parametric Non-linear Flow and Multi-scale Fusioncited by 110
  2. 2022IFRNet: Intermediate Feature Refine Network for Efficient Frame Interpolationcited by 107
  3. 2021Asymmetric Bilateral Motion Estimation for Video Frame Interpolationcited by 101
  4. 2022VideoINR: Learning Video Implicit Neural Representation for Continuous Space-Time Super-Resolutioncited by 101
  5. 2021Geometry-Free View Synthesis: Transformers and no 3D Priorscited by 99
  6. 2023Extracting Motion and Appearance via Inter-Frame Attention for Efficient Video Frame Interpolationcited by 97
  7. 2024Dynamic Gaussian Marbles for Novel View Synthesis of Casual Monocular Videoscited by 96
  8. 2023Generative Image Dynamicscited by 93
  9. 2023LDMVFI: Video Frame Interpolation with Latent Diffusion Modelscited by 88
  10. 2022InfiniteNature-Zero: Learning Perpetual View Generation of Natural Scenes from Single Imagescited by 82
  11. 2023VideoFlow: Exploiting Temporal Cues for Multi-frame Optical Flow Estimationcited by 82
  12. 2023StyleRF: Zero-shot 3D Style Transfer of Neural Radiance Fieldscited by 79
  13. 2024StyleGaussian: Instant 3D Style Transfer with Gaussian Splattingcited by 78
  14. 2024GenWarp: Single Image to Novel Views with Semantic-Preserving Generative Warpingcited by 72
  15. 2024ToonCrafter: Generative Cartoon Interpolationcited by 68
  16. 2023Forward Flow for Novel View Synthesis of Dynamic Scenescited by 67
  17. 2024Video Interpolation with Diffusion Modelscited by 67
  18. 2021Deep Animation Video Interpolation in the Wildcited by 65
  19. 2025UniK3D: Universal Camera Monocular 3D Estimationcited by 60
  20. 2022Unifying Motion Deblurring and Frame Interpolation with Eventscited by 58
  21. 2021CDFI: Compression-Driven Network Design for Frame Interpolationcited by 55
  22. 2024MemFlow: Optical Flow Estimation and Prediction with Memorycited by 55
  23. 2021Temporal Modulation Network for Controllable Space-Time Video Super-Resolutioncited by 54
  24. 2021Stochastic Image-to-Video Synthesis using cINNscited by 49
  25. 2021AdaPool: Exponential Adaptive Pooling for Information-Retaining Downsamplingcited by 47
  26. 2021Controllable Animation of Fluid Elements in Still Imagescited by 44
  27. 2022RSTT: Real-time Spatial Temporal Transformer for Space-Time Video Super-Resolutioncited by 44
  28. 2021iPOKE: Poking a Still Image for Controlled Stochastic Video Synthesiscited by 42
  29. 2022Many-to-many Splatting for Efficient Video Frame Interpolationcited by 42
  30. 2024Generative Inbetweening: Adapting Image-to-Video Models for Keyframe Interpolationcited by 42
  31. 2025PhysAnimator: Physics-Guided Generative Cartoon Animationcited by 40
  32. 2022A Unified Pyramid Recurrent Network for Video Frame Interpolationcited by 39
  33. 2021Understanding Object Dynamics for Interactive Image-to-Video Synthesiscited by 38
  34. 2024Magic Fixup: Streamlining Photo Editing by Watching Dynamic Videoscited by 38
  35. 2023Enhancing Video Super-Resolution via Implicit Resampling-based Alignmentcited by 35
  36. 2021ST-MFNet: A Spatio-Temporal Multi-Flow Network for Frame Interpolationcited by 34
  37. 2022FloLPIPS: A Bespoke Video Quality Metric for Frame Interpoationcited by 34
  38. 2024TANGO: Co-Speech Gesture Video Reenactment with Hierarchical Audio Motion Embedding and Diffusion Interpolationcited by 32

50 most cited of 276 All 276 (JSON)