mlsanityData to 7 Oct 2026

TransFlow: Transformer as Flow Learner

arXiv 2304.11523 ↗First posted 23 Apr 2023cs.CVcited by 32

Not on the map. Why?

Abstract (arXiv)

Optical flow is an indispensable building block for various important computer vision tasks, including motion estimation, object tracking, and disparity measurement. In this work, we propose TransFlow, a pure transformer architecture for optical flow estimation. Compared to dominant CNN-based methods, TransFlow demonstrates three advantages. First, it provides more accurate correlation and trustworthy matching in flow estimation by utilizing spatial self-attention and cross-attention mechanisms between adjacent frames to effectively capture global dependencies; Second, it recovers more compromised information (e.g., occlusion and motion blur) in flow estimation through long-range temporal association in dynamic scenes; Third, it enables a concise self-learning paradigm and effectively eliminate the complex and laborious multi-stage pre-training procedures. We achieve the state-of-the-art results on the Sintel, KITTI-15, as well as several downstream tasks, including video object detection, interpolation and stabilization. For its efficacy, we hope TransFlow could serve as a flexible baseline for optical flow estimation.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). Builds on 47 of its 61 references that are in the map. First posted on arXiv 23 Apr 2023.

Builds on

47 of 61 references.

References · 47

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  5. 2020End-to-End Object Detection with Transformerscited by 7,384
  6. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  7. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  8. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  9. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
  10. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,096
  11. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,054
  12. 2017PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volumecited by 966
35 more
  1. 2021CvT: Introducing Convolutions to Vision Transformerscited by 946
  2. 2015A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimationcited by 878
  3. 2021Uformer: A General U-Shaped Transformer for Image Restorationcited by 864
  4. 2016FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networkscited by 780
  5. 2016Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurringcited by 722
  6. 2021Conditional Positional Encodings for Vision Transformerscited by 402
  7. 2017Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolationcited by 321
  8. 2022FlowFormer: A Transformer Architecture for Optical Flowcited by 273
  9. 2021Learning to Estimate Hidden Motions with Global Motion Aggregationcited by 223
  10. 2019Pay Less Attention with Lightweight and Dynamic Convolutionscited by 211
  11. 2020AANet: Adaptive Aggregation Network for Efficient Stereo Matchingcited by 202
  12. 2021E-RAFT: Dense Optical Flow from Event Camerascited by 149
  13. 2020MaskFlownet: Asymmetric Feature Matching with Learnable Occlusion Maskcited by 117
  14. 2022IFRNet: Intermediate Feature Refine Network for Efficient Frame Interpolationcited by 107
  15. 2020Learning Feature Descriptors using Camera Pose Supervisioncited by 104
  16. 2021FuseFormer: Fusing Fine-Grained Information in Transformers for Video Inpaintingcited by 94
  17. 2018Hierarchical Discrete Distribution Decomposition for Match Density Estimationcited by 93
  18. 2019Sequence Level Semantics Aggregation for Video Object Detectioncited by 87
  19. 2022CRAFT: Cross-Attentional Flow Transformer for Robust Optical Flowcited by 82
  20. 2019Relation Distillation Networks for Video Object Detectioncited by 77
  21. 2020SoftSort: A Continuous Relaxation for the argsort Operatorcited by 66
  22. 2021Lite Vision Transformer with Enhanced Self-Attentioncited by 64
  23. 2022Visual Recognition with Deep Nearest Centroidscited by 59
  24. 2022Uniform Masking: Enabling MAE Pre-training for Pyramid-based Vision Transformers with Localitycited by 57
  25. 2022Physical Attack on Monocular Depth Estimation with Optimal Adversarial Patchescited by 57
  26. 2021High-Resolution Optical Flow from 1D Attention and Correlationcited by 54
  27. 2021Learning Optical Flow from a Few Matchescited by 53
  28. 2020Displacement-Invariant Matching Cost Learning for Accurate Optical Flow Estimationcited by 49
  29. 2020UPFlow: Upsampling Pyramid for Unsupervised Optical Flow Learningcited by 49
  30. 2022Disentangling Object Motion and Occlusion for Unsupervised Multi-frame Monocular Depthcited by 48
  31. 2021TF-Blender: Temporal Feature Blender for Video Object Detectioncited by 37
  32. 2022Learning Optical Flow with Adaptive Graph Reasoningcited by 33
  33. 2022PTSEFormer: Progressive Temporal-Spatial Enhanced TransFormer Towards Video Object Detectioncited by 17
  34. 2022Learning Equivariant Segmentation with Instance-Unique Queryingcited by 17
  35. 2022Towards Unbiased Label Distribution Learning for Facial Pose Estimation Using Anisotropic Spherical Gaussiancited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 6; 2024: 15; 2025: 8; 2026: 3.

32 papers.

Sort:

By area:

  1. 2024SEA-RAFT: Simple, Efficient, Accurate RAFT for Optical Flowcited by 169
  2. 2023E^2VPT: An Effective and Efficient Approach for Visual Prompt Tuningcited by 70
  3. 2025DPFlow: Adaptive Optical Flow Estimation with a Dual-Pyramid Frameworkcited by 30
  4. 2023StreamFlow: Streamlined Multi-Frame Optical Flow Estimation for Video Sequencescited by 12
  5. 2024Rethinking RAFT for Efficient Optical Flowcited by 8
  6. 2023ClusterFormer: Clustering As A Universal Visual Learnercited by 6
  7. 2023Unsupervised Optical Flow Estimation with Dynamic Timing Representation for Spike Cameracited by 5
  8. 2024Self-supervised Adversarial Training of Monocular Depth Estimation against Physical-World Attackscited by 5
  9. 2024Exploring the Common Appearance-Boundary Adaptation for Nighttime Optical Flowcited by 4
  10. 2024Prototypical Transformer as Unified Motion Learnerscited by 4
  11. 2024Adverse Weather Optical Flow: Cumulative Homogeneous-Heterogeneous Adaptationcited by 4
  12. 2024BadPart: Unified Black-box Adversarial Patch Attacks against Pixel-wise Regression Taskscited by 3
20 more
  1. 2024CMR-Agent: Learning a Cross-Modal Agent for Iterative Image-to-Point Cloud Registrationcited by 3
  2. 2024Benchmarking the Robustness of Optical Flow Estimation to Corruptionscited by 3
  3. 2024Independently Keypoint Learning for Small Object Semantic Correspondencecited by 2
  4. 2025A Survey on Video Analytics in Cloud-Edge-Terminal Collaborative Systemscited by 2
  5. 2025MambaFlow: A Mamba-Centric Architecture for End-to-End Optical Flow Estimationcited by 2
  6. 2025ProbDiffFlow: An Efficient Learning-Free Framework for Probabilistic Single-Image Optical Flow Estimationcited by 2
  7. 2025Towards Efficient Real-Time Video Motion Transfer via Generative Time Series Modelingcited by 2
  8. 2023FLODCAST: Flow and Depth Forecasting via Multimodal Recurrent Architecturescited by 1
  9. 2024Radiance Field Learners As UAV First-Person Viewerscited by 1
  10. 2024SDformerFlow: Spatiotemporal swin spikeformer for event-based optical flow estimationcited by 1
  11. 2026Generative Modeling in Protein Design: Neural Representations, Conditional Generation, and Evaluation Standardscited by 1
  12. 2023Lightweight Improved Residual Network for Efficient Inverse Tone Mappingcited by 0
  13. 2024Occlusion-Aware 3D Motion Interpretation for Abnormal Behavior Detectioncited by 0
  14. 2024Leveraging Object Priors for Point Trackingcited by 0
  15. 2024Content-Aware Preserving Image Generationcited by 0
  16. 2025Adept: Annotation-Denoising Auxiliary Tasks with Discrete Cosine Transform Map and Keypoint for Human-Centric Pretrainingcited by 0
  17. 2025Intrinsic Dimensionality as a Model-Free Measure of Class Imbalancecited by 0
  18. 2025Flow and Depth Assisted Video Prediction with Latent Transformercited by 0
  19. 2026RAFT-MSF++: Temporal Geometry-Motion Feature Fusion for Self-Supervised Monocular Scene Flowcited by 0
  20. 2026FreeFlow: A Bias-free Hierarchical Transformer for Optical Flow Estimationcited by 0