mlsanityData to 7 Oct 2026

Extracting Motion and Appearance via Inter-Frame Attention for Efficient Video Frame Interpolation

arXiv 2303.00440 ↗First posted 1 Mar 2023cs.CVcited by 97

Not on the map. Why?

Abstract (arXiv)

Effectively extracting inter-frame motion and appearance information is important for video frame interpolation (VFI). Previous works either extract both types of information in a mixed way or elaborate separate modules for each type of information, which lead to representation ambiguity and low efficiency. In this paper, we propose a novel module to explicitly extract motion and appearance information via a unifying operation. Specifically, we rethink the information process in inter-frame attention and reuse its attention map for both appearance feature enhancement and motion information extraction. Furthermore, for efficient VFI, our proposed module could be seamlessly integrated into a hybrid CNN and Transformer architecture. This hybrid pipeline can alleviate the computational complexity of inter-frame attention as well as preserve detailed low-level structure information. Experimental results demonstrate that, for both fixed- and arbitrary-timestep interpolation, our method achieves state-of-the-art performance on various datasets. Meanwhile, our approach enjoys a lighter computation overhead over models with close performance. The source code and models are available at https://github.com/MCG-NJU/EMA-VFI.

Key facts. Cited by 97 papers in the map (data through 7 Oct 2026). Builds on 47 of its 62 references that are in the map. First posted on arXiv 1 Mar 2023.

Builds on

47 of 62 references.

References · 47

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  4. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  5. 2017Decoupled Weight Decay Regularizationcited by 1,722
  6. 2016Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Networkcited by 1,199
  7. 2015Multi-Scale Context Aggregation by Dilated Convolutionscited by 1,128
  8. 2021CvT: Introducing Convolutions to Vision Transformerscited by 946
  9. 2021PVT v2: Improved Baselines with Pyramid Vision Transformercited by 749
  10. 2017Video Enhancement with Task-Oriented Flowcited by 612
  11. 2021CoAtNet: Marrying Convolution and Attention for All Data Sizescited by 566
  12. 2021ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biasescited by 484
35 more
  1. 2021Conditional Positional Encodings for Vision Transformerscited by 402
  2. 2021Early Convolutions Help Transformers See Bettercited by 395
  3. 2017Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolationcited by 321
  4. 2020Softmax Splatting for Video Frame Interpolationcited by 276
  5. 2021Incorporating Convolution Designs into Visual Transformerscited by 248
  6. 2019Depth-Aware Video Frame Interpolationcited by 232
  7. 2021LocalViT: Analyzing Locality in Vision Transformerscited by 228
  8. 2017Video Frame Interpolation via Adaptive Separable Convolutioncited by 211
  9. 2017Video Frame Synthesis using Deep Voxel Flowcited by 209
  10. 2015DeepStereo: Learning to Predict New Views from the World's Imagerycited by 185
  11. 2017Video Frame Interpolation via Adaptive Convolutioncited by 158
  12. 2018Context-aware Synthesis for Video Frame Interpolationcited by 153
  13. 2022FILM: Frame Interpolation for Large Motioncited by 149
  14. 2018Video Compression through Image Interpolationcited by 143
  15. 2019AdaCoF: Adaptive Collaboration of Flows for Video Frame Interpolationcited by 133
  16. 2021XVFI: eXtreme Video Frame Interpolationcited by 123
  17. 2021Video Frame Interpolation Transformercited by 122
  18. 2020BMBC:Bilateral Motion Estimation with Bilateral Cost Volume for Video Interpolationcited by 117
  19. 2018MEMC-Net: Motion Estimation and Motion Compensation Driven Neural Network for Video Interpolation and Enhancementcited by 111
  20. 2022IFRNet: Intermediate Feature Refine Network for Efficient Frame Interpolationcited by 107
  21. 2021Asymmetric Bilateral Motion Estimation for Video Frame Interpolationcited by 101
  22. 2020FLAVR: Flow-Agnostic Video Representations for Fast Frame Interpolationcited by 97
  23. 2019Video Modeling with Correlation Networkscited by 73
  24. 2020MotionSqueeze: Neural Motion Feature Learning for Video Understandingcited by 72
  25. 2020Multiple Video Frame Interpolation via Enhanced Deformable Separable Convolutioncited by 62
  26. 2017Appearance-and-Relation Networks for Video Classificationcited by 61
  27. 2021CDFI: Compression-Driven Network Design for Frame Interpolationcited by 55
  28. 2022Many-to-many Splatting for Efficient Video Frame Interpolationcited by 42
  29. 2016Learning Image Matching by Simply Watching Videocited by 41
  30. 2018Spatio-Temporal Channel Correlation Networks for Action Classificationcited by 38
  31. 2021ST-MFNet: A Spatio-Temporal Multi-Flow Network for Frame Interpolationcited by 34
  32. 2020Revisiting Adaptive Convolutions for Video Frame Interpolationcited by 32
  33. 2022Neighbor Correspondence Matching for Flow-based Video Frame Synthesiscited by 12
  34. 2022Convolutional Embedding Makes Hierarchical Vision Transformer Strongercited by 10
  35. 2021Enhanced Correlation Matching based Video Frame Interpolationcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 15; 2024: 34; 2025: 33; 2026: 15.

97 papers.

Sort:

By area:

  1. 2023AMT: All-Pairs Multi-Field Transforms for Efficient Frame Interpolationcited by 177
  2. 2024ToonCrafter: Generative Cartoon Interpolationcited by 68
  3. 2024Allegro: Open the Black Box of Commercial-Level Video Generation Modelcited by 68
  4. 2024Video Interpolation with Diffusion Modelscited by 67
  5. 2023A Dynamic Multi-Scale Voxel Flow Network for Video Predictioncited by 53
  6. 2024MagicVideo-V2: Multi-Stage High-Aesthetic Video Generationcited by 48
  7. 2023MGMAE: Motion Guided Masking for Video Masked Autoencodingcited by 40
  8. 2024Sparse Global Matching for Video Frame Interpolation with Large Motioncited by 27
  9. 2024AWT: Transferring Vision-Language Models via Augmentation, Weighting, and Transportationcited by 27
  10. 2024DS-NeRV: Implicit Neural Video Representation with Decomposed Static and Dynamic Codescited by 25
  11. 2024Perception-Oriented Video Frame Interpolation via Asymmetric Blendingcited by 23
  12. 2024Repurposing Pre-trained Video Diffusion Models for Event-based Video Interpolationcited by 23
38 more
  1. 2024Generative Inbetweening through Frame-wise Conditions-Driven Video Generationcited by 23
  2. 2024VFIMamba: Video Frame Interpolation with State Space Modelscited by 22
  3. 2025EDEN: Enhanced Diffusion for High-quality Large-motion Video Frame Interpolationcited by 19
  4. 2024Dual DETRs for Multi-Label Temporal Action Detectioncited by 18
  5. 2025PMQ-VE: Progressive Multi-Frame Quantization for Video Enhancementcited by 17
  6. 2024DreamMover: Leveraging the Prior of Diffusion Models for Image Interpolation with Large Motioncited by 16
  7. 2024Data-Efficient Unsupervised Interpolation Without Any Intermediate Frame for 4D Medical Imagescited by 15
  8. 2023ZeroI2V: Zero-Cost Adaptation of Pre-trained Transformers from Image to Videocited by 14
  9. 2024BiM-VFI: Bidirectional Motion Field-Guided Frame Interpolation for Video with Non-uniform Motionscited by 13
  10. 2025Motion-Aware Generative Frame Interpolationcited by 13
  11. 2024Long Video Diffusion Generation with Segmented Cross-Attention and Content-Rich Video Data Curationcited by 12
  12. 2023SportsSloMo: A New Benchmark and Baselines for Human-centric Video Frame Interpolationcited by 11
  13. 2024OCAI: Improving Optical Flow Estimation by Occlusion and Consistency Aware Interpolationcited by 11
  14. 2024LAVIB: A Large-scale Video Interpolation Benchmarkcited by 11
  15. 2024Disentangled Motion Modeling for Video Frame Interpolationcited by 11
  16. 2023Video Frame Interpolation with Stereo Event and Intensity Cameracited by 10
  17. 2026OneVision-Encoder: Codec-Aligned Sparsity as a Foundational Principle for Multimodal Intelligencecited by 10
  18. 2025BF-STVSR: B-Splines and Fourier-Best Friends for High Fidelity Spatial-Temporal Video Super-Resolutioncited by 9
  19. 2023Scale-Adaptive Feature Aggregation for Efficient Space-Time Video Super-Resolutioncited by 8
  20. 2025Ev-3DOD: Pushing the Temporal Boundaries of 3D Object Detection with Event Camerascited by 8
  21. 2025AceVFI: A Comprehensive Survey of Advances in Video Frame Interpolationcited by 8
  22. 2025Hierarchical Flow Diffusion for Efficient Frame Interpolationcited by 7
  23. 2025TLB-VFI: Temporal-Aware Latent Brownian Bridge Diffusion for Video Frame Interpolationcited by 7
  24. 2025Versatile Transition Generation with Image-to-Video Diffusioncited by 7
  25. 2025Trokens: Semantic-Aware Relational Trajectory Tokens for Few-Shot Action Recognitioncited by 7
  26. 2024ZeroSmooth: Training-free Diffuser Adaptation for High Frame Rate Video Generationcited by 5
  27. 2024Grid: Omni Visual Generationcited by 5
  28. 2025Bridge Frame and Event: Common Spatiotemporal Fusion for High-Dynamic Scene Optical Flowcited by 5
  29. 2025FB-Diff: Fourier Basis-guided Diffusion for Temporal Interpolation of 4D Medical Imagingcited by 5
  30. 2024Neural Rendering and Its Hardware Acceleration: A Reviewcited by 4
  31. 2024Dual-Camera Smooth Zoom on Mobile Phonescited by 4
  32. 2024MoSt-DSA: Modeling Motion and Structural Interactions for Direct Multi-Frame Interpolation in DSA Imagescited by 4
  33. 2024Thin-Plate Spline-based Interpolation for Animation Line Inbetweeningcited by 4
  34. 2024FastPCI: Motion-Structure Guided Fast Point Cloud Frame Interpolationcited by 4
  35. 2025Progressive Growing of Video Tokenizers for Temporally Compact Latent Spacescited by 4
  36. 2026Towards Holistic Modeling for Video Frame Interpolation with Auto-regressive Diffusion Transformerscited by 4
  37. 2023Video Face Re-Aging: Toward Temporally Consistent Face Re-Agingcited by 3
  38. 2024Continuous Sign Language Recognition Based on Motor attention mechanism and frame-level Self-distillationcited by 3

50 most cited of 97 All 97 (JSON)