# High-Resolution Optical Flow from 1D Attention and Correlation

arXiv: https://arxiv.org/abs/2104.13918 · HTML: https://mlsanity.com/paper/2104.13918/

## Key facts

- Cited by 54 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 47 references that are in the map.
- First posted on arXiv 28 Apr 2021.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://mlsanity.com/paper/1901.02860/) (2019)
- [Generating Long Sequences with Sparse Transformers](https://mlsanity.com/paper/1904.10509/) (2019)
- [Linformer: Self-Attention with Linear Complexity](https://mlsanity.com/paper/2006.04768/) (2020)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [Two-Stream Convolutional Networks for Action Recognition in Videos](https://mlsanity.com/paper/1406.2199/) (2014)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [The 2017 DAVIS Challenge on Video Object Segmentation](https://mlsanity.com/paper/1704.00675/) (2017)
- [A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation](https://mlsanity.com/paper/1512.02134/) (2015)
- [Efficient Transformers: A Survey](https://mlsanity.com/paper/2009.06732/) (2020)
- [FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks](https://mlsanity.com/paper/1612.01925/) (2016)
- [CCNet: Criss-Cross Attention for Semantic Segmentation](https://mlsanity.com/paper/1811.11721/) (2018)
- [Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation](https://mlsanity.com/paper/1712.00080/) (2017)
- [Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentation](https://mlsanity.com/paper/2003.07853/) (2020)
- [AANet: Adaptive Aggregation Network for Efficient Stereo Matching](https://mlsanity.com/paper/2004.09548/) (2020)
- [Iterative Residual Refinement for Joint Optical Flow and Occlusion Estimation](https://mlsanity.com/paper/1904.05290/) (2019)
- [Learning the Depths of Moving People by Watching Frozen People](https://mlsanity.com/paper/1904.11111/) (2019)
- [MaskFlownet: Asymmetric Feature Matching with Learnable Occlusion Mask](https://mlsanity.com/paper/2003.10955/) (2020)
- [Models Matter, So Does Training: An Empirical Study of CNNs for Optical Flow Estimation](https://mlsanity.com/paper/1809.05571/) (2018)
- [Hierarchical Discrete Distribution Decomposition for Match Density Estimation](https://mlsanity.com/paper/1812.06264/) (2018)
- [EpicFlow: Edge-Preserving Interpolation of Correspondences for Optical Flow](https://mlsanity.com/paper/1501.02565/) (2015)
- [Accurate Optical Flow via Direct Cost Volume Processing](https://mlsanity.com/paper/1704.07325/) (2017)
- [Full Flow: Optical Flow Estimation By Global Optimization over Regular Grids](https://mlsanity.com/paper/1604.03513/) (2016)
- [Devon: Deformable Volume Network for Learning Optical Flow](https://mlsanity.com/paper/1802.07351/) (2018)

## Built on by (most cited first, 30 of 54)

- [CoTracker: It is Better to Track Together](https://mlsanity.com/paper/2307.07635/) (2023)
- [GMFlow: Learning Optical Flow via Global Matching](https://mlsanity.com/paper/2111.13680/) (2021)
- [FlowFormer: A Transformer Architecture for Optical Flow](https://mlsanity.com/paper/2203.16194/) (2022)
- [Unifying Flow, Stereo and Depth Estimation](https://mlsanity.com/paper/2211.05783/) (2022)
- [AMT: All-Pairs Multi-Field Transforms for Efficient Frame Interpolation](https://mlsanity.com/paper/2304.09790/) (2023)
- [FlowFormer++: Masked Cost Volume Autoencoding for Pretraining Optical Flow Estimation](https://mlsanity.com/paper/2303.01237/) (2023)
- [VideoFlow: Exploiting Temporal Cues for Multi-frame Optical Flow Estimation](https://mlsanity.com/paper/2303.08340/) (2023)
- [Motion-Guided Latent Diffusion for Temporally Consistent Real-world Video Super-resolution](https://mlsanity.com/paper/2312.00853/) (2023)
- [SKFlow: Learning Optical Flow with Super Kernels](https://mlsanity.com/paper/2205.14623/) (2022)
- [Dense Continuous-Time Optical Flow from Events and Frames](https://mlsanity.com/paper/2203.13674/) (2022)
- [Implicit Warping for Animation with Image Sets](https://mlsanity.com/paper/2210.01794/) (2022)
- [TAPTR: Tracking Any Point with Transformers as Detection](https://mlsanity.com/paper/2403.13042/) (2024)
- [DIP: Deep Inverse Patchmatch for High-Resolution Optical Flow](https://mlsanity.com/paper/2204.00330/) (2022)
- [Disentangling Architecture and Training for Optical Flow](https://mlsanity.com/paper/2203.10712/) (2022)
- [TMA: Temporal Motion Aggregation for Event-based Optical Flow](https://mlsanity.com/paper/2303.11629/) (2023)
- [TransFlow: Transformer as Flow Learner](https://mlsanity.com/paper/2304.11523/) (2023)
- [Lightweight Event-based Optical Flow Estimation via Iterative Deblurring](https://mlsanity.com/paper/2211.13726/) (2022)
- [DPFlow: Adaptive Optical Flow Estimation with a Dual-Pyramid Framework](https://mlsanity.com/paper/2503.14880/) (2025)
- [PanoFlow: Learning 360deg Optical Flow for Surrounding Temporal Understanding](https://mlsanity.com/paper/2202.13388/) (2022)
- [GAFlow: Incorporating Gaussian Attention into Optical Flow](https://mlsanity.com/paper/2309.16217/) (2023)
- [TAPTRv2: Attention-based Position Update Improves Tracking Any Point](https://mlsanity.com/paper/2407.16291/) (2024)
- [AnyFlow: Arbitrary Scale Optical Flow with Implicit Neural Representation](https://mlsanity.com/paper/2303.16493/) (2023)
- [CSFlow: Learning Optical Flow via Cross Strip Correlation for Autonomous Driving](https://mlsanity.com/paper/2202.00909/) (2022)
- [DistractFlow: Improving Optical Flow Estimation via Realistic Distractions and Pseudo-Labeling](https://mlsanity.com/paper/2303.14078/) (2023)
- [Track Everything Everywhere Fast and Robustly](https://mlsanity.com/paper/2403.17931/) (2024)
- [Self-supervised AutoFlow](https://mlsanity.com/paper/2212.01762/) (2022)
- [Multi-Scale RAFT: Combining Hierarchical Concepts for Learning-based Optical FLow Estimation](https://mlsanity.com/paper/2207.12163/) (2022)
- [SemARFlow: Injecting Semantics into Unsupervised Optical Flow Estimation for Autonomous Driving](https://mlsanity.com/paper/2303.06209/) (2023)
- [Memory-Efficient Optical Flow via Radius-Distribution Orthogonal Cost Volume](https://mlsanity.com/paper/2312.03790/) (2023)
- [Rethinking RAFT for Efficient Optical Flow](https://mlsanity.com/paper/2401.00833/) (2024)

## Abstract (arXiv)

Optical flow is inherently a 2D search problem, and thus the computational complexity grows quadratically with respect to the search window, making large displacements matching infeasible for high-resolution images. In this paper, we take inspiration from Transformers and propose a new method for high-resolution optical flow estimation with significantly less computation. Specifically, a 1D attention operation is first applied in the vertical direction of the target image, and then a simple 1D correlation in the horizontal direction of the attended image is able to achieve 2D correspondence modeling effect. The directions of attention and correlation can also be exchanged, resulting in two 3D cost volumes that are concatenated for optical flow estimation. The novel 1D formulation empowers our method to scale to very high-resolution input images while maintaining competitive performance. Extensive experiments on Sintel, KITTI and real-world 4K (2160 × 3840) resolution images demonstrated the effectiveness and superiority of our proposed method. Code and models are available at urlhttps://github.com/haofeixu/flow1d.