# CRAFT: Cross-Attentional Flow Transformer for Robust Optical Flow

arXiv: https://arxiv.org/abs/2203.16896 · HTML: https://mlsanity.com/paper/2203.16896/

## Key facts

- Cited by 82 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 49 references that are in the map.
- First posted on arXiv 31 Mar 2022.

## Builds on (most cited first, 30 of 37)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation](https://mlsanity.com/paper/1512.02134/) (2015)
- [FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks](https://mlsanity.com/paper/1612.01925/) (2016)
- [Perceiver IO: A General Architecture for Structured Inputs & Outputs](https://mlsanity.com/paper/2107.14795/) (2021)
- [Optical Flow Estimation using a Spatial Pyramid Network](https://mlsanity.com/paper/1611.00850/) (2016)
- [RAFT-Stereo: Multilevel Recurrent Field Transforms for Stereo Matching](https://mlsanity.com/paper/2109.07547/) (2021)
- [Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation](https://mlsanity.com/paper/1712.00080/) (2017)
- [How Do Vision Transformers Work?](https://mlsanity.com/paper/2202.06709/) (2022)
- [Self-supervised Co-training for Video Representation Learning](https://mlsanity.com/paper/2010.09709/) (2020)
- [Learning to Estimate Hidden Motions with Global Motion Aggregation](https://mlsanity.com/paper/2104.02409/) (2021)
- [Self-supervised Video Object Segmentation by Motion Grouping](https://mlsanity.com/paper/2104.07658/) (2021)
- [Iterative Residual Refinement for Joint Optical Flow and Occlusion Estimation](https://mlsanity.com/paper/1904.05290/) (2019)
- [SelFlow: Self-Supervised Learning of Optical Flow](https://mlsanity.com/paper/1904.09117/) (2019)
- [MaskFlownet: Asymmetric Feature Matching with Learnable Occlusion Mask](https://mlsanity.com/paper/2003.10955/) (2020)
- [Position Information in Transformers: An Overview](https://mlsanity.com/paper/2102.11090/) (2021)
- [AutoFlow: Learning a Better Training Set for Optical Flow](https://mlsanity.com/paper/2104.14544/) (2021)
- [Actor-Centric Relation Network](https://mlsanity.com/paper/1807.10982/) (2018)
- [Optical Flow Guided Feature: A Fast and Robust Motion Representation for Video Action Recognition](https://mlsanity.com/paper/1711.11152/) (2017)
- [SMURF: Self-Teaching Multi-Frame Unsupervised RAFT with Full-Image Warping](https://mlsanity.com/paper/2105.07014/) (2021)
- [Accurate Optical Flow via Direct Cost Volume Processing](https://mlsanity.com/paper/1704.07325/) (2017)
- [Medical Image Segmentation Using Squeeze-and-Expansion Transformers](https://mlsanity.com/paper/2105.09511/) (2021)
- [Displacement-Invariant Matching Cost Learning for Accurate Optical Flow Estimation](https://mlsanity.com/paper/2010.14851/) (2020)

## Built on by (most cited first, 30 of 82)

- [CoTracker: It is Better to Track Together](https://mlsanity.com/paper/2307.07635/) (2023)
- [Unifying Flow, Stereo and Depth Estimation](https://mlsanity.com/paper/2211.05783/) (2022)
- [SEA-RAFT: Simple, Efficient, Accurate RAFT for Optical Flow](https://mlsanity.com/paper/2405.14793/) (2024)
- [CroCo v2: Improved Cross-view Completion Pre-training for Stereo Matching and Optical Flow](https://mlsanity.com/paper/2211.10408/) (2022)
- [The Surprising Effectiveness of Diffusion Models for Optical Flow and Monocular Depth Estimation](https://mlsanity.com/paper/2306.01923/) (2023)
- [FlowFormer++: Masked Cost Volume Autoencoding for Pretraining Optical Flow Estimation](https://mlsanity.com/paper/2303.01237/) (2023)
- [CompletionFormer: Depth Completion with Convolutions and Vision Transformers](https://mlsanity.com/paper/2304.13030/) (2023)
- [VideoFlow: Exploiting Temporal Cues for Multi-frame Optical Flow Estimation](https://mlsanity.com/paper/2303.08340/) (2023)
- [Dense Optical Tracking: Connecting the Dots](https://mlsanity.com/paper/2312.00786/) (2023)
- [UFM: A Simple Path towards Unified Dense Correspondence with Flow](https://mlsanity.com/paper/2506.09278/) (2025)
- [WAFT: Warping-Alone Field Transforms for Optical Flow](https://mlsanity.com/paper/2506.21526/) (2025)
- [Rethinking Optical Flow from Geometric Matching Consistent Perspective](https://mlsanity.com/paper/2303.08384/) (2023)
- [TransFlow: Transformer as Flow Learner](https://mlsanity.com/paper/2304.11523/) (2023)
- [DeFlow: Decoder of Scene Flow Network in Autonomous Driving](https://mlsanity.com/paper/2401.16122/) (2024)
- [Learning Optical Flow from Event Camera with Rendered Dataset](https://mlsanity.com/paper/2303.11011/) (2023)
- [DropMAE: Learning Representations via Masked Autoencoders with Spatial-Attention Dropout for Temporal Matching Tasks](https://mlsanity.com/paper/2304.00571/) (2023)
- [Zero-Shot Monocular Scene Flow Estimation in the Wild](https://mlsanity.com/paper/2501.10357/) (2025)
- [TorchDEQ: A Library for Deep Equilibrium Models](https://mlsanity.com/paper/2310.18605/) (2023)
- [AccFlow: Backward Accumulation for Long-Range Optical Flow](https://mlsanity.com/paper/2308.13133/) (2023)
- [GAFlow: Incorporating Gaussian Attention into Optical Flow](https://mlsanity.com/paper/2309.16217/) (2023)
- [AnyFlow: Arbitrary Scale Optical Flow with Implicit Neural Representation](https://mlsanity.com/paper/2303.16493/) (2023)
- [Shape-Constraint Recurrent Flow for 6D Object Pose Estimation](https://mlsanity.com/paper/2306.13266/) (2023)
- [SAMFlow: Eliminating Any Fragmentation in Optical Flow with Segment Anything Model](https://mlsanity.com/paper/2307.16586/) (2023)
- [CCMR: High Resolution Optical Flow Estimation via Coarse-to-Fine Context-Guided Motion Reasoning](https://mlsanity.com/paper/2311.02661/) (2023)
- [RIAV-MVS: Recurrent-Indexing an Asymmetric Volume for Multi-View Stereo](https://mlsanity.com/paper/2205.14320/) (2022)
- [Video Unsupervised Domain Adaptation with Deep Learning: A Comprehensive Survey](https://mlsanity.com/paper/2211.10412/) (2022)
- [Win-Win: Training High-Resolution Vision Transformers from Two Windows](https://mlsanity.com/paper/2310.00632/) (2023)
- [StreamFlow: Streamlined Multi-Frame Optical Flow Estimation for Video Sequences](https://mlsanity.com/paper/2311.17099/) (2023)
- [LayeredFlow: A Real-World Benchmark for Non-Lambertian Multi-Layer Optical Flow](https://mlsanity.com/paper/2409.05688/) (2024)
- [Self-supervised AutoFlow](https://mlsanity.com/paper/2212.01762/) (2022)

## Abstract (arXiv)

Optical flow estimation aims to find the 2D motion field by identifying corresponding pixels between two images. Despite the tremendous progress of deep learning-based optical flow methods, it remains a challenge to accurately estimate large displacements with motion blur. This is mainly because the correlation volume, the basis of pixel matching, is computed as the dot product of the convolutional features of the two images. The locality of convolutional features makes the computed correlations susceptible to various noises. On large displacements with motion blur, noisy correlations could cause severe errors in the estimated flow. To overcome this challenge, we propose a new architecture "CRoss-Attentional Flow Transformer" (CRAFT), aiming to revitalize the correlation volume computation. In CRAFT, a Semantic Smoothing Transformer layer transforms the features of one frame, making them more global and semantically stable. In addition, the dot-product correlations are replaced with transformer Cross-Frame Attention. This layer filters out feature noises through the Query and Key projections, and computes more accurate correlations. On Sintel (Final) and KITTI (foreground) benchmarks, CRAFT has achieved new state-of-the-art performance. Moreover, to test the robustness of different models on large motions, we designed an image shifting attack that shifts input images to generate large artificial motions. Under this attack, CRAFT performs much more robustly than two representative methods, RAFT and GMA. The code of CRAFT is is available at https://github.com/askerlee/craft.