mlsanityData to 7 Oct 2026

AMT: All-Pairs Multi-Field Transforms for Efficient Frame Interpolation

arXiv 2304.09790 ↗First posted 19 Apr 2023cs.CVcited by 177

Not on the map. Why?

Cited by 70 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present All-Pairs Multi-Field Transforms (AMT), a new network architecture for video frame interpolation. It is based on two essential designs. First, we build bidirectional correlation volumes for all pairs of pixels, and use the predicted bilateral flows to retrieve correlations for updating both flows and the interpolated content feature. Second, we derive multiple groups of fine-grained flow fields from one pair of updated coarse flows for performing backward warping on the input frames separately. Combining these two designs enables us to generate promising task-oriented flows and reduce the difficulties in modeling large motions and handling occluded areas during frame interpolation. These qualities promote our model to achieve state-of-the-art performance on various benchmarks with high efficiency. Moreover, our convolution-based model competes favorably compared to Transformer-based models in terms of accuracy and efficiency. Our code is available at https://github.com/MCG-NKU/AMT.

Key facts. Cited by 177 papers in the map (data through 7 Oct 2026). Builds on 63 of its 85 references that are in the map. First posted on arXiv 19 Apr 2023.

Builds on

63 of 85 references.

References · 63

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  5. 2017Squeeze-and-Excitation Networkscited by 4,763
  6. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  7. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  8. 2017Deformable Convolutional Networkscited by 1,481
  9. 2016Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Networkcited by 1,199
  10. 2016Instance Normalization: The Missing Ingredient for Fast Stylizationcited by 1,187
  11. 2017PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volumecited by 966
  12. 2020Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenescited by 750
38 more
  1. 2016Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurringcited by 722
  2. 2017Video Enhancement with Task-Oriented Flowcited by 612
  3. 2017End-to-End Learning of Geometry and Context for Deep Stereo Regressioncited by 359
  4. 2021RAFT-Stereo: Multilevel Recurrent Field Transforms for Stereo Matchingcited by 348
  5. 2021BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignmentcited by 340
  6. 2021GMFlow: Learning Optical Flow via Global Matchingcited by 324
  7. 2017Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolationcited by 321
  8. 2020Softmax Splatting for Video Frame Interpolationcited by 276
  9. 2022FlowFormer: A Transformer Architecture for Optical Flowcited by 273
  10. 2019Depth-Aware Video Frame Interpolationcited by 232
  11. 2021Learning to Estimate Hidden Motions with Global Motion Aggregationcited by 223
  12. 2017Video Frame Interpolation via Adaptive Separable Convolutioncited by 211
  13. 2020Real-Time Intermediate Flow Estimation for Video Frame Interpolationcited by 211
  14. 2017Video Frame Synthesis using Deep Voxel Flowcited by 209
  15. 2017UnFlow: Unsupervised Learning of Optical Flow with a Bidirectional Census Losscited by 202
  16. 2015DeepStereo: Learning to Predict New Views from the World's Imagerycited by 185
  17. 2016View Synthesis by Appearance Flowcited by 172
  18. 2017Video Frame Interpolation via Adaptive Convolutioncited by 158
  19. 2018Context-aware Synthesis for Video Frame Interpolationcited by 153
  20. 2022FILM: Frame Interpolation for Large Motioncited by 149
  21. 2018Video Compression through Image Interpolationcited by 143
  22. 2019Quadratic video interpolationcited by 143
  23. 2019AdaCoF: Adaptive Collaboration of Flows for Video Frame Interpolationcited by 133
  24. 2019Iterative Residual Refinement for Joint Optical Flow and Occlusion Estimationcited by 131
  25. 2021XVFI: eXtreme Video Frame Interpolationcited by 123
  26. 2021Video Frame Interpolation Transformercited by 122
  27. 2020RAFT-3D: Scene Flow using Rigid-Motion Embeddingscited by 121
  28. 2020BMBC:Bilateral Motion Estimation with Bilateral Cost Volume for Video Interpolationcited by 117
  29. 2018MEMC-Net: Motion Estimation and Motion Compensation Driven Neural Network for Video Interpolation and Enhancementcited by 111
  30. 2022Towards An End-to-End Framework for Flow-Guided Video Inpaintingcited by 111
  31. 2022IFRNet: Intermediate Feature Refine Network for Efficient Frame Interpolationcited by 107
  32. 2021Asymmetric Bilateral Motion Estimation for Video Frame Interpolationcited by 101
  33. 2020FLAVR: Flow-Agnostic Video Representations for Fast Frame Interpolationcited by 97
  34. 2023Extracting Motion and Appearance via Inter-Frame Attention for Efficient Video Frame Interpolationcited by 97
  35. 2020Zooming Slow-Mo: Fast and Accurate One-Stage Space-Time Video Super-Resolutioncited by 91
  36. 2021Deep Animation Video Interpolation in the Wildcited by 65
  37. 2022Global Matching with Overlapping Attention for Optical Flow Estimationcited by 64
  38. 2020Multiple Video Frame Interpolation via Enhanced Deformable Separable Convolutioncited by 62

50 most cited of 63 All 63 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 10; 2024: 57; 2025: 65; 2026: 45.

177 papers.

Sort:

By area:

  1. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
  2. 2024Taming Rectified Flow for Inversion and Editingcited by 200
  3. 2024MiraData: A Large-Scale Video Dataset with Long Durations and Structured Captionscited by 144
  4. 2024SV4D: Dynamic 3D Content Generation with Multi-Frame and Multi-View Consistencycited by 143
  5. 2024ConsistI2V: Enhancing Visual Consistency for Image-to-Video Generationcited by 130
  6. 2024VideoDPO: Omni-Preference Alignment for Video Diffusion Generationcited by 102
  7. 2024FreeLong: Training-Free Long Video Generation with SpectralBlend Temporal Attentioncited by 87
  8. 2024T2V-Turbo: Breaking the Quality Bottleneck of Video Consistency Model with Mixed Reward Feedbackcited by 84
  9. 2023Free-Bloom: Zero-Shot Text-to-Video Generator with LLM Director and LDM Animatorcited by 74
  10. 2024Video Interpolation with Diffusion Modelscited by 67
  11. 2023GPT4Motion: Scripting Physical Motions in Text-to-Video Generation via Blender-Oriented GPT Planningcited by 61
  12. 2024COVE: Unleashing the Diffusion Feature Correspondence for Consistent Video Editingcited by 59
38 more
  1. 2023Dense Optical Tracking: Connecting the Dotscited by 58
  2. 2024Animate3D: Animating Any 3D Model with Multi-view Video Diffusioncited by 55
  3. 2024T2V-Turbo-v2: Enhancing Video Generation Model Post-Training through Data, Reward, and Conditional Guidance Designcited by 54
  4. 2025Video-Bench: Human-Aligned Video Generation Benchmarkcited by 53
  5. 2024MagicTime: Time-lapse Video Generation Models as Metamorphic Simulatorscited by 51
  6. 2024MarDini: Masked Autoregressive Diffusion for Video Generation at Scalecited by 49
  7. 2025VMBench: A Benchmark for Perception-Aligned Video Motion Generationcited by 48
  8. 2024StyleMaster: Stylize Your Video with Artistic Generation and Translationcited by 47
  9. 2023CorrMatch: Label Propagation via Correlation Matching for Semi-Supervised Semantic Segmentationcited by 43
  10. 2024Track4Gen: Teaching Video Diffusion Models to Track Points Improves Video Generationcited by 42
  11. 2024Mora: Enabling Generalist Video Generation via A Multi-Agent Frameworkcited by 41
  12. 2024DreamPhysics: Learning Physics-Based 3D Dynamics with Video Diffusion Priorscited by 40
  13. 2025Inference-Time Text-to-Video Alignment with Diffusion Latent Beam Searchcited by 40
  14. 2024EgoVid-5M: A Large-Scale Video-Action Dataset for Egocentric Video Generationcited by 38
  15. 2024MovieBench: A Hierarchical Movie Level Dataset for Long Video Generationcited by 35
  16. 2025Robotic Manipulation by Imitating Generated Videos Without Physical Demonstrationscited by 33
  17. 2024TANGO: Co-Speech Gesture Video Reenactment with Hierarchical Audio Motion Embedding and Diffusion Interpolationcited by 32
  18. 2025VideoShield: Regulating Diffusion-based Video Generation Models via Watermarkingcited by 32
  19. 2025AnimateAnyMesh: A Feed-Forward 4D Foundation Model for Text-Driven Universal Mesh Animationcited by 31
  20. 2024GAIA: Rethinking Action Quality Assessment for AI-Generated Videoscited by 30
  21. 2025PAI-Bench: A Comprehensive Benchmark For Physical AIcited by 30
  22. 2024Sparse Global Matching for Video Frame Interpolation with Large Motioncited by 27
  23. 2025VideoScene: Distilling Video Diffusion Model to Generate 3D Scenes in One Stepcited by 27
  24. 2025ContextFlow: Training-Free Video Object Editing via Adaptive Context Enrichmentcited by 27
  25. 2025IVEBench: Modern Benchmark Suite for Instruction-Guided Video Editing Assessmentcited by 27
  26. 2025Pack and Force Your Memory: Long-form and Consistent Video Generationcited by 26
  27. 2024Framer: Interactive Frame Interpolationcited by 24
  28. 2024Perception-Oriented Video Frame Interpolation via Asymmetric Blendingcited by 23
  29. 2024Generative Inbetweening through Frame-wise Conditions-Driven Video Generationcited by 23
  30. 2025VideoGen-Eval: Agent-based System for Video Generation Evaluationcited by 23
  31. 2024VFIMamba: Video Frame Interpolation with State Space Modelscited by 22
  32. 2024HOI-Swap: Swapping Objects in Videos with Hand-Object Interaction Awarenesscited by 21
  33. 2024Frame Interpolation with Consecutive Brownian Bridge Diffusioncited by 19
  34. 2024ViBiDSampler: Enhancing Video Interpolation Using Bidirectional Diffusion Samplercited by 19
  35. 2025EDEN: Enhanced Diffusion for High-quality Large-motion Video Frame Interpolationcited by 19
  36. 2024Generalizable Implicit Motion Modeling for Video Frame Interpolationcited by 16
  37. 2024MotionStone: Decoupled Motion Intensity Modulation with Diffusion Transformer for Image-to-Video Generationcited by 16
  38. 2024Bora: Biomedical Generalist Video Generation Modelcited by 15

50 most cited of 177 All 177 (JSON)