mlsanityData to 7 Oct 2026

GMFlow: Learning Optical Flow via Global Matching

arXiv 2111.13680 ↗First posted 26 Nov 2021cs.CVcited by 324

Not on the map. Why?

Cited by 32 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Learning-based optical flow estimation has been dominated with the pipeline of cost volume with convolutions for flow regression, which is inherently limited to local correlations and thus is hard to address the long-standing challenge of large displacements. To alleviate this, the state-of-the-art framework RAFT gradually improves its prediction quality by using a large number of iterative refinements, achieving remarkable performance but introducing linearly increasing inference time. To enable both high accuracy and efficiency, we completely revamp the dominant flow regression pipeline by reformulating optical flow as a global matching problem, which identifies the correspondences by directly comparing feature similarities. Specifically, we propose a GMFlow framework, which consists of three main components: a customized Transformer for feature enhancement, a correlation and softmax layer for global feature matching, and a self-attention layer for flow propagation. We further introduce a refinement step that reuses GMFlow at higher feature resolution for residual flow prediction. Our new framework outperforms 31-refinements RAFT on the challenging Sintel benchmark, while using only one refinement and running faster, suggesting a new paradigm for accurate and efficient optical flow estimation. Code is available at https://github.com/haofeixu/gmflow.

Key facts. Cited by 324 papers in the map (data through 7 Oct 2026). Builds on 40 of its 54 references that are in the map. First posted on arXiv 26 Nov 2021.

Builds on

40 of 54 references.

References · 40

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  5. 2020End-to-End Object Detection with Transformerscited by 7,384
  6. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  7. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  8. 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,437
  9. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,054
  10. 2021LoFTR: Detector-Free Local Feature Matching with Transformerscited by 974
  11. 2017PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volumecited by 966
  12. 2015A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimationcited by 878
28 more
  1. 2016FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networkscited by 780
  2. 2021Perceiver IO: A General Architecture for Structured Inputs & Outputscited by 616
  3. 2020TartanAir: A Dataset to Push the Limits of Visual SLAMcited by 494
  4. 2021ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biasescited by 484
  5. 2016Optical Flow Estimation using a Spatial Pyramid Networkcited by 421
  6. 2017End-to-End Learning of Geometry and Context for Deep Stereo Regressioncited by 359
  7. 2020Virtual KITTI 2cited by 345
  8. 2016Virtual Worlds as Proxy for Multi-Object Tracking Analysiscited by 340
  9. 2021Learning to Estimate Hidden Motions with Global Motion Aggregationcited by 223
  10. 2021ViTAE: Vision Transformer Advanced by Exploring Intrinsic Inductive Biascited by 216
  11. 2017UnFlow: Unsupervised Learning of Optical Flow with a Bidirectional Census Losscited by 202
  12. 2020AANet: Adaptive Aggregation Network for Efficient Stereo Matchingcited by 202
  13. 2020Revisiting Stereo Depth Estimation From a Sequence-to-Sequence Perspective with Transformerscited by 190
  14. 2021COTR: Correspondence Transformer for Matching Across Imagescited by 182
  15. 2017Playing for Benchmarkscited by 169
  16. 2019Scale-Aware Trident Networks for Object Detectioncited by 159
  17. 2019GLU-Net: Global-Local Universal Network for Dense Flow and Correspondencescited by 139
  18. 2019Iterative Residual Refinement for Joint Optical Flow and Occlusion Estimationcited by 131
  19. 2020Learning Feature Descriptors using Camera Pose Supervisioncited by 104
  20. 2021AutoFlow: Learning a Better Training Set for Optical Flowcited by 102
  21. 2018Models Matter, So Does Training: An Empirical Study of CNNs for Optical Flow Estimationcited by 94
  22. 2018Hierarchical Discrete Distribution Decomposition for Match Density Estimationcited by 93
  23. 2015EpicFlow: Edge-Preserving Interpolation of Correspondences for Optical Flowcited by 80
  24. 2021High-Resolution Optical Flow from 1D Attention and Correlationcited by 54
  25. 2020Displacement-Invariant Matching Cost Learning for Accurate Optical Flow Estimationcited by 49
  26. 2018Learning Rigidity in Dynamic Scenes with a Moving Camera for 3D Motion Field Estimationcited by 38
  27. 2020LiteFlowNet3: Resolving Correspondence Ambiguity for More Accurate Optical Flow Estimationcited by 37
  28. 2015Flow Fields: Dense Correspondence Fields for Highly Accurate Large Displacement Optical Flow Estimationcited by 22

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 17; 2023: 79; 2024: 97; 2025: 87; 2026: 44.

324 papers.

Sort:

By area:

  1. 2024MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Imagescited by 553
  2. 2023VideoComposer: Compositional Video Synthesis with Motion Controllabilitycited by 461
  3. 2023TAPIR: Tracking Any Point with per-frame Initialization and temporal Refinementcited by 281
  4. 2024DepthSplat: Connecting Gaussian Splatting and Depthcited by 275
  5. 2022Unifying Flow, Stereo and Depth Estimationcited by 270
  6. 2023Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translationcited by 253
  7. 2024Shape of Motion: 4D Reconstruction from a Single Videocited by 247
  8. 2025Video Depth Anything: Consistent Depth Estimation for Super-Long Videoscited by 202
  9. 2023AMT: All-Pairs Multi-Field Transforms for Efficient Frame Interpolationcited by 177
  10. 2023Spring: A High-Resolution High-Detail Dataset and Benchmark for Scene Flow, Optical Flow and Stereocited by 170
  11. 2024SEA-RAFT: Simple, Efficient, Accurate RAFT for Optical Flowcited by 169
  12. 2022CroCo v2: Improved Cross-view Completion Pre-training for Stereo Matching and Optical Flowcited by 165
38 more
  1. 2023NICER-SLAM: Neural Implicit Scene Encoding for RGB SLAMcited by 162
  2. 2023Learning to Act from Actionless Videos through Dense Correspondencescited by 159
  3. 2024GenAD: Generalized Predictive Model for Autonomous Drivingcited by 150
  4. 2023The Surprising Effectiveness of Diffusion Models for Optical Flow and Monocular Depth Estimationcited by 116
  5. 2023FlowFormer++: Masked Cost Volume Autoencoding for Pretraining Optical Flow Estimationcited by 114
  6. 2025DriveSuprim: Towards Precise Trajectory Selection for End-to-End Planningcited by 91
  7. 2023VideoFlow: Exploiting Temporal Cues for Multi-frame Optical Flow Estimationcited by 82
  8. 2025Vision-Language-Action Models for Robotics: A Review Towards Real-World Applicationscited by 82
  9. 2023Segment Anything Meets Point Trackingcited by 81
  10. 2023FlowVid: Taming Imperfect Optical Flows for Consistent Video-to-Video Synthesiscited by 79
  11. 2024How NeRFs and 3D Gaussian Splatting are Reshaping SLAM: a Surveycited by 72
  12. 2024TranSplat: Generalizable 3D Gaussian Splatting from Sparse Multi-View Images with Transformerscited by 71
  13. 2023MuRF: Multi-Baseline Radiance Fieldscited by 70
  14. 2024DINO-Tracker: Taming DINO for Self-Supervised Point Tracking in a Single Videocited by 64
  15. 2024DreamScene4D: Dynamic Multi-Object Scene Generation from Monocular Videoscited by 63
  16. 2024Image Conductor: Precision Control for Interactive Video Synthesiscited by 62
  17. 2024MotionGS: Exploring Explicit Motion Guidance for Deformable 3D Gaussian Splattingcited by 62
  18. 2024DELTA: Dense Efficient Long-range 3D Tracking for any videocited by 62
  19. 2025InsViE-1M: Effective Instruction-based Video Editing with Elaborate Dataset Constructioncited by 62
  20. 2024FlowMap: High-Quality Camera Poses, Intrinsics, and Depth via Gradient Descentcited by 61
  21. 2024PF3plat: Pose-Free Feed-Forward 3D Gaussian Splattingcited by 60
  22. 2023Dense Optical Tracking: Connecting the Dotscited by 58
  23. 2024MemFlow: Optical Flow Estimation and Prediction with Memorycited by 55
  24. 2024HiSplat: Hierarchical 3D Gaussian Splatting for Generalizable Sparse-View Reconstructioncited by 55
  25. 2025AllTracker: Efficient Dense Point Tracking at High Resolutioncited by 54
  26. 2023A Dynamic Multi-Scale Voxel Flow Network for Video Predictioncited by 53
  27. 2024FRESCO: Spatial-Temporal Correspondence for Zero-Shot Video Translationcited by 48
  28. 2023Explicit Correspondence Matching for Generalizable Neural Radiance Fieldscited by 46
  29. 2024GFlow: Recovering 4D World from Monocular Videocited by 46
  30. 2024Diffusion Model-Based Video Editing: A Surveycited by 44
  31. 2024MonoOcc: Digging into Monocular Semantic Occupancy Predictioncited by 43
  32. 2025Dynamic Point Maps: A Versatile Representation for Dynamic 3D Reconstructioncited by 43
  33. 2024TC-Bench: Benchmarking Temporal Compositionality in Text-to-Video and Image-to-Video Generationcited by 42
  34. 2024STRAP: Robot Sub-Trajectory Retrieval for Augmented Policy Learningcited by 42
  35. 2024Towards Generalist Robot Learning from Internet Video: A Surveycited by 40
  36. 2024Motion Guidance: Diffusion-Based Image Editing with Differentiable Motion Estimatorscited by 39
  37. 2023Unifying Correspondence, Pose and NeRF for Pose-Free Novel View Synthesis from Stereo Pairscited by 37
  38. 2025Instant Gaussian Stream: Fast and Generalizable Streaming of Dynamic Scene Reconstruction via Gaussian Splattingcited by 35

50 most cited of 324 All 324 (JSON)