mlsanityData to 14 Oct 2026

FlowFormer: A Transformer Architecture for Optical Flow

arXiv 2203.16194 ↗First posted 30 Mar 2022cs.CVcited by 275

Not on the map. Why?

Cited by 20 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We introduce optical Flow transFormer, dubbed as FlowFormer, a transformer-based neural network architecture for learning optical flow. FlowFormer tokenizes the 4D cost volume built from an image pair, encodes the cost tokens into a cost memory with alternate-group transformer (AGT) layers in a novel latent space, and decodes the cost memory via a recurrent transformer decoder with dynamic positional cost queries. On the Sintel benchmark, FlowFormer achieves 1.159 and 2.088 average end-point-error (AEPE) on the clean and final pass, a 16.5% and 15.5% error reduction from the best published result (1.388 and 2.47). Besides, FlowFormer also achieves strong generalization performance. Without being trained on Sintel, FlowFormer achieves 1.01 AEPE on the clean pass of Sintel training set, outperforming the best published result (1.29) by 21.7%.

Key facts. Cited by 275 papers in the map (data through 14 Oct 2026). Builds on 46 of its 63 references that are in the map. First posted on arXiv 30 Mar 2022.

Builds on

46 of 63 references.

References · 46

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  5. 2020End-to-End Object Detection with Transformerscited by 7,414
  6. 2022A ConvNet for the 2020scited by 3,579
  7. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,415
  8. 2021SwinIR: Image Restoration Using Swin Transformercited by 1,887
  9. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,850
  10. 2020Point Transformercited by 1,470
  11. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,446
  12. 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,443
34 more
  1. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,059
  2. 2021LoFTR: Detector-Free Local Feature Matching with Transformerscited by 984
  3. 2017PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volumecited by 968
  4. 2020Pre-Trained Image Processing Transformercited by 893
  5. 2015A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimationcited by 881
  6. 2016FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networkscited by 784
  7. 2020PCT: Point cloud transformercited by 732
  8. 2021Perceiver IO: A General Architecture for Structured Inputs & Outputscited by 629
  9. 2017Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolutioncited by 453
  10. 2016Optical Flow Estimation using a Spatial Pyramid Networkcited by 424
  11. 2020BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyondcited by 304
  12. 2021Learning to Estimate Hidden Motions with Global Motion Aggregationcited by 224
  13. 2021COTR: Correspondence Transformer for Matching Across Imagescited by 183
  14. 2020Learning Joint Spatial-Temporal Transformations for Video Inpaintingcited by 163
  15. 2018Frame-Recurrent Video Super-Resolutioncited by 160
  16. 2019Quadratic video interpolationcited by 145
  17. 2020MaskFlownet: Asymmetric Feature Matching with Learnable Occlusion Maskcited by 117
  18. 2019Deep Flow-Guided Video Inpaintingcited by 116
  19. 2019Deep Video Inpaintingcited by 109
  20. 2020Flow-edge Guided Video Completioncited by 104
  21. 2021AutoFlow: Learning a Better Training Set for Optical Flowcited by 102
  22. 2021FuseFormer: Fusing Fine-Grained Information in Transformers for Video Inpaintingcited by 95
  23. 2018Models Matter, So Does Training: An Empirical Study of CNNs for Optical Flow Estimationcited by 94
  24. 2018Hierarchical Discrete Distribution Decomposition for Match Density Estimationcited by 93
  25. 2017Optical Flow Guided Feature: A Fast and Robust Motion Representation for Video Action Recognitioncited by 82
  26. 2021CATs: Cost Aggregation Transformers for Visual Correspondencecited by 77
  27. 2019A Lightweight Optical Flow CNN - Revisiting Data Fidelity and Regularizationcited by 76
  28. 2021High-Resolution Optical Flow from 1D Attention and Correlationcited by 54
  29. 2021Learning Optical Flow from a Few Matchescited by 53
  30. 2018Representation Flow for Action Recognitioncited by 51
  31. 2022RNNPose: Recurrent 6-DoF Object Pose Refinement with Robust Correspondence Field Estimation and Pose Optimizationcited by 38
  32. 2021VS-Net: Voting with Segmentation for Visual Localizationcited by 29
  33. 2020Optical Flow in Dense Foggy Scenes using Semi-Supervised Learningcited by 20
  34. 2021LIFE: Lighting Invariant Flow Estimationcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 16; 2023: 69; 2024: 82; 2025: 67; 2026: 41.

275 papers.

Sort:

By area:

  1. 2024Grounding Image Matching in 3D with MASt3Rcited by 983
  2. 2023One Fits All:Power General Time Series Analysis by Pretrained LMcited by 632
  3. 2023CoTracker: It is Better to Track Togethercited by 522
  4. 2023Human Preference Score: Better Aligning Text-to-Image Models with Human Preferencecited by 287
  5. 2022Unifying Flow, Stereo and Depth Estimationcited by 275
  6. 2022SegNeXt: Rethinking Convolutional Attention Design for Semantic Segmentationcited by 252
  7. 2024Shape of Motion: 4D Reconstruction from a Single Videocited by 250
  8. 2023CAT-Seg: Cost Aggregation for Open-Vocabulary Semantic Segmentationcited by 219
  9. 2024Motion-I2V: Consistent and Controllable Image-to-Video Generation with Explicit Motion Modelingcited by 211
  10. 2024SpatialTracker: Tracking Any 2D Pixels in 3D Spacecited by 181
  11. 2023AMT: All-Pairs Multi-Field Transforms for Efficient Frame Interpolationcited by 180
  12. 2024SEA-RAFT: Simple, Efficient, Accurate RAFT for Optical Flowcited by 174
38 more
  1. 2023Spring: A High-Resolution High-Detail Dataset and Benchmark for Scene Flow, Optical Flow and Stereocited by 172
  2. 2022CroCo v2: Improved Cross-view Completion Pre-training for Stereo Matching and Optical Flowcited by 166
  3. 2023The Surprising Effectiveness of Diffusion Models for Optical Flow and Monocular Depth Estimationcited by 116
  4. 2024TimeMixer++: A General Time Series Pattern Machine for Universal Predictive Analysiscited by 116
  5. 2023FlowFormer++: Masked Cost Volume Autoencoding for Pretraining Optical Flow Estimationcited by 114
  6. 2023VideoControlNet: A Motion-Guided Video-to-Video Translation Framework by Using Diffusion Model with ControlNetcited by 83
  7. 2023VideoFlow: Exploiting Temporal Cues for Multi-frame Optical Flow Estimationcited by 82
  8. 2022A Simple Baseline for Video Restoration with Grouped Spatial-temporal Shiftcited by 68
  9. 2024DELTA: Dense Efficient Long-range 3D Tracking for any videocited by 65
  10. 2024DINO-Tracker: Taming DINO for Self-Supervised Point Tracking in a Single Videocited by 64
  11. 2024MotionGS: Exploring Explicit Motion Guidance for Deformable 3D Gaussian Splattingcited by 63
  12. 2023MFT: Long-Term Tracking of Every Pixelcited by 62
  13. 2023Dense Optical Tracking: Connecting the Dotscited by 61
  14. 2024MemFlow: Optical Flow Estimation and Prediction with Memorycited by 55
  15. 2022MVSFormer: Multi-View Stereo by Learning Robust Image Features and Temperature-based Depthcited by 53
  16. 2023A Dynamic Multi-Scale Voxel Flow Network for Video Predictioncited by 53
  17. 2024MVSFormer++: Revealing the Devil in Transformer's Details for Multi-View Stereocited by 50
  18. 2024TAPTR: Tracking Any Point with Transformers as Detectioncited by 45
  19. 2025Dynamic Point Maps: A Versatile Representation for Dynamic 3D Reconstructioncited by 44
  20. 2022textttGradICON: Approximate Diffeomorphisms via Gradient Inverse Consistencycited by 41
  21. 2023A survey on efficient vision transformers: algorithms, techniques, and performance benchmarkingcited by 41
  22. 2024Motion Consistency Model: Accelerating Video Diffusion with Disentangled Motion-Appearance Distillationcited by 39
  23. 2025Context-Alignment: Activating and Enhancing LLM Capabilities in Time Seriescited by 39
  24. 2023DynPoint: Dynamic Neural Point For View Synthesiscited by 37
  25. 2023Unifying Correspondence, Pose and NeRF for Pose-Free Novel View Synthesis from Stereo Pairscited by 37
  26. 2024AMEGO: Active Memory from long EGOcentric videoscited by 36
  27. 2025UFM: A Simple Path towards Unified Dense Correspondence with Flowcited by 36
  28. 2024DiffIR2VR-Zero: Zero-Shot Video Restoration with Diffusion-based Image Restoration Modelscited by 34
  29. 2025WAFT: Warping-Alone Field Transforms for Optical Flowcited by 34
  30. 2023PATS: Patch Area Transportation with Subdivision for Local Feature Matchingcited by 33
  31. 2023BlinkFlow: A Dataset to Push the Limits of Event-based Optical Flow Estimationcited by 33
  32. 2023Rethinking Optical Flow from Geometric Matching Consistent Perspectivecited by 33
  33. 2022Neural Matching Fields: Implicit Representation of Matching Fields for Visual Correspondencecited by 32
  34. 2023TransFlow: Transformer as Flow Learnercited by 32
  35. 2022Lightweight Event-based Optical Flow Estimation via Iterative Deblurringcited by 31
  36. 2024HDRFlow: Real-Time HDR Video Reconstruction with Large Motionscited by 31
  37. 2024Splatter a Video: Video Gaussian Representation for Versatile Processingcited by 31
  38. 2024InstantDrag: Improving Interactivity in Drag-based Image Editingcited by 31

50 most cited of 275 All 275 (JSON)