mlsanityData to 7 Oct 2026

Video Frame Interpolation Transformer

arXiv 2111.13817 ↗First posted 27 Nov 2021cs.CVcited by 122

Not on the map. Why?

Cited by 21 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Existing methods for video interpolation heavily rely on deep convolution neural networks, and thus suffer from their intrinsic limitations, such as content-agnostic kernel weights and restricted receptive field. To address these issues, we propose a Transformer-based video interpolation framework that allows content-aware aggregation weights and considers long-range dependencies with the self-attention operations. To avoid the high computational cost of global self-attention, we introduce the concept of local attention into video interpolation and extend it to the spatial-temporal domain. Furthermore, we propose a space-time separation strategy to save memory usage, which also improves performance. In addition, we develop a multi-scale frame synthesis scheme to fully realize the potential of Transformers. Extensive experiments demonstrate the proposed model performs favorably against the state-of-the-art methods both quantitatively and qualitatively on a variety of benchmark datasets.

Key facts. Cited by 122 papers in the map (data through 7 Oct 2026). Builds on 44 of its 55 references that are in the map. First posted on arXiv 27 Nov 2021.

Builds on

44 of 55 references.

References · 44

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  6. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  7. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  8. 2020End-to-End Object Detection with Transformerscited by 7,384
  9. 2016Layer Normalizationcited by 5,579
  10. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  11. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  12. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
32 more
  1. 2017Non-local Neural Networkscited by 2,290
  2. 2021SwinIR: Image Restoration Using Swin Transformercited by 1,869
  3. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,609
  4. 2016Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Networkcited by 1,199
  5. 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,139
  6. 2021Multiscale Vision Transformerscited by 896
  7. 2020Pre-Trained Image Processing Transformercited by 891
  8. 2017Video Enhancement with Task-Oriented Flowcited by 612
  9. 2019Stand-Alone Self-Attention in Vision Modelscited by 542
  10. 2017Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classificationcited by 521
  11. 2020Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentationcited by 326
  12. 2017Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolationcited by 321
  13. 2020Softmax Splatting for Video Frame Interpolationcited by 276
  14. 2019Depth-Aware Video Frame Interpolationcited by 232
  15. 2017Video Frame Interpolation via Adaptive Separable Convolutioncited by 211
  16. 2019Local Relation Networks for Image Recognitioncited by 210
  17. 2017Video Frame Interpolation via Adaptive Convolutioncited by 158
  18. 2019Recurrent Back-Projection Network for Video Super-Resolutioncited by 147
  19. 2019Quadratic video interpolationcited by 143
  20. 2019AdaCoF: Adaptive Collaboration of Flows for Video Frame Interpolationcited by 133
  21. 2021XVFI: eXtreme Video Frame Interpolationcited by 123
  22. 2021Video Super-Resolution Transformercited by 120
  23. 2020BMBC:Bilateral Motion Estimation with Bilateral Cost Volume for Video Interpolationcited by 117
  24. 2020FLAVR: Flow-Agnostic Video Representations for Fast Frame Interpolationcited by 97
  25. 2020Zooming Slow-Mo: Fast and Accurate One-Stage Space-Time Video Super-Resolutioncited by 91
  26. 2020Multiple Video Frame Interpolation via Enhanced Deformable Separable Convolutioncited by 62
  27. 2021Temporal Modulation Network for Controllable Space-Time Video Super-Resolutioncited by 54
  28. 2020All at Once: Temporally Adaptive Multi-Frame Interpolation with Advanced Motion Modelingcited by 42
  29. 2018Learning to Synthesize Motion Blurcited by 37
  30. 20213D Human Texture Estimation from a Single Image with Transformerscited by 35
  31. 2020Revisiting Adaptive Convolutions for Video Frame Interpolationcited by 32
  32. 2021Learning Spatial and Spatio-Temporal Pixel Aggregations for Image and Video Denoisingcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 16; 2023: 34; 2024: 36; 2025: 26; 2026: 10.

122 papers.

Sort:

By area:

  1. 2023AMT: All-Pairs Multi-Field Transforms for Efficient Frame Interpolationcited by 177
  2. 2023Extracting Motion and Appearance via Inter-Frame Attention for Efficient Video Frame Interpolationcited by 97
  3. 2023LDMVFI: Video Frame Interpolation with Latent Diffusion Modelscited by 88
  4. 2023Feature Shrinkage Pyramid for Camouflaged Object Detection with Transformerscited by 87
  5. 2022A Unified Pyramid Recurrent Network for Video Frame Interpolationcited by 39
  6. 2022FloLPIPS: A Bespoke Video Quality Metric for Frame Interpoationcited by 34
  7. 2024TANGO: Co-Speech Gesture Video Reenactment with Hierarchical Audio Motion Embedding and Diffusion Interpolationcited by 32
  8. 2023TBP-Former: Learning Temporal Bird's-Eye-View Pyramid for Joint Perception and Prediction in Vision-Centric Autonomous Drivingcited by 30
  9. 2024Sparse Global Matching for Video Frame Interpolation with Large Motioncited by 27
  10. 2024DS-NeRV: Implicit Neural Video Representation with Decomposed Static and Dynamic Codescited by 25
  11. 2024Framer: Interactive Frame Interpolationcited by 24
  12. 2025Event-Based Video Frame Interpolation With Cross-Modal Asymmetric Bidirectional Motion Fieldscited by 24
38 more
  1. 2023BiFormer: Learning Bilateral Motion Estimation via Bilateral Transformer for 4K Video Frame Interpolationcited by 23
  2. 2024Perception-Oriented Video Frame Interpolation via Asymmetric Blendingcited by 23
  3. 2024Repurposing Pre-trained Video Diffusion Models for Event-based Video Interpolationcited by 23
  4. 2024VFIMamba: Video Frame Interpolation with State Space Modelscited by 22
  5. 2023AccFlow: Backward Accumulation for Long-Range Optical Flowcited by 19
  6. 2024ViBiDSampler: Enhancing Video Interpolation Using Bidirectional Diffusion Samplercited by 19
  7. 2025EDEN: Enhanced Diffusion for High-quality Large-motion Video Frame Interpolationcited by 19
  8. 2023SRFormerV2: Taking a Closer Look at Permuted Self-Attention for Image Super-Resolutioncited by 18
  9. 2022Exploring Motion Ambiguity and Alignment for High-Quality Video Frame Interpolationcited by 17
  10. 2022Enhanced Bi-directional Motion Estimation for Video Frame Interpolationcited by 17
  11. 2025PMQ-VE: Progressive Multi-Frame Quantization for Video Enhancementcited by 17
  12. 2024Generalizable Implicit Motion Modeling for Video Frame Interpolationcited by 16
  13. 2024DreamMover: Leveraging the Prior of Diffusion Models for Image Interpolation with Large Motioncited by 16
  14. 2022A Perceptual Quality Metric for Video Frame Interpolationcited by 15
  15. 2023FastLLVE: Real-Time Low-Light Video Enhancement with Intensity-Aware Lookup Tablecited by 15
  16. 2022TTVFI: Learning Trajectory-Aware Transformer for Video Frame Interpolationcited by 14
  17. 2024Motion-aware Latent Diffusion Models for Video Frame Interpolationcited by 14
  18. 2024KMTalk: Speech-Driven 3D Facial Animation with Key Motion Embeddingcited by 14
  19. 2023NeuralPCI: Spatio-temporal Neural Field for 3D Point Cloud Multi-frame Non-linear Interpolationcited by 13
  20. 2024BiM-VFI: Bidirectional Motion Field-Guided Frame Interpolation for Video with Non-uniform Motionscited by 13
  21. 2025Motion-Aware Generative Frame Interpolationcited by 13
  22. 2023Continuous Intermediate Token Learning with Implicit Motion Manifold for Keyframe Based Motion Interpolationcited by 12
  23. 2023Light-VQA: A Multi-Dimensional Quality Assessment Model for Low-Light Video Enhancementcited by 12
  24. 2023SportsSloMo: A New Benchmark and Baselines for Human-centric Video Frame Interpolationcited by 11
  25. 2023MaskINT: Video Editing via Interpolative Non-autoregressive Masked Transformerscited by 11
  26. 2024OCAI: Improving Optical Flow Estimation by Occlusion and Consistency Aware Interpolationcited by 11
  27. 2024Disentangled Motion Modeling for Video Frame Interpolationcited by 11
  28. 2024FuXi-2.0: Advancing machine learning weather forecasting model for practical applicationscited by 11
  29. 2022Cross-Attention Transformer for Video Interpolationcited by 10
  30. 2023Video Frame Interpolation with Stereo Event and Intensity Cameracited by 10
  31. 2023Deep Geometrized Cartoon Line Inbetweeningcited by 9
  32. 2024SMPLer: Taming Transformers for Monocular 3D Human Shape and Pose Estimationcited by 9
  33. 2025TimeTracker: Event-based Continuous Point Tracking for Video Frame Interpolation with Non-linear Motioncited by 9
  34. 2023Towards Nonlinear-Motion-Aware and Occlusion-Robust Rolling Shutter Correctioncited by 8
  35. 2023Video Frame Interpolation with Densely Queried Bilateral Correlationcited by 8
  36. 2023Scale-Adaptive Feature Aggregation for Efficient Space-Time Video Super-Resolutioncited by 8
  37. 2024From Sim-to-Real: Toward General Event-based Low-light Frame Interpolation with Per-scene Optimizationcited by 8
  38. 2025AceVFI: A Comprehensive Survey of Advances in Video Frame Interpolationcited by 8

50 most cited of 122 All 122 (JSON)