# FastFlowNet: A Lightweight Network for Fast Optical Flow Estimation

arXiv: https://arxiv.org/abs/2103.04524 · HTML: https://mlsanity.com/paper/2103.04524/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 32 references that are in the map.
- First posted on arXiv 8 Mar 2021.

## Builds on (most cited first, 20 of 20)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices](https://mlsanity.com/paper/1707.01083/) (2017)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation](https://mlsanity.com/paper/1512.02134/) (2015)
- [FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks](https://mlsanity.com/paper/1612.01925/) (2016)
- [Optical Flow Estimation using a Spatial Pyramid Network](https://mlsanity.com/paper/1611.00850/) (2016)
- [Iterative Residual Refinement for Joint Optical Flow and Occlusion Estimation](https://mlsanity.com/paper/1904.05290/) (2019)
- [MaskFlownet: Asymmetric Feature Matching with Learnable Occlusion Mask](https://mlsanity.com/paper/2003.10955/) (2020)
- [Hierarchical Discrete Distribution Decomposition for Match Density Estimation](https://mlsanity.com/paper/1812.06264/) (2018)
- [Fast Optical Flow using Dense Inverse Search](https://mlsanity.com/paper/1603.03590/) (2016)
- [Anytime Stereo Image Depth Estimation on Mobile Devices](https://mlsanity.com/paper/1810.11408/) (2018)
- [Real-Time Semantic Stereo Matching](https://mlsanity.com/paper/1910.00541/) (2019)
- [Improved Optical Flow for Gesture-based Human-robot Interaction](https://mlsanity.com/paper/1905.08685/) (2019)
- [FDFlowNet: Fast Optical Flow Estimation using a Deep Lightweight Network](https://mlsanity.com/paper/2006.12263/) (2020)
- [Correlation Flow: Robust Optical Flow Using Kernel Cross-Correlators](https://mlsanity.com/paper/1802.07078/) (2018)
- [Unsupervised Motion Representation Enhanced Network for Action Recognition](https://mlsanity.com/paper/2103.03465/) (2021)

## Built on by (most cited first, 28 of 28)

- [IFRNet: Intermediate Feature Refine Network for Efficient Frame Interpolation](https://mlsanity.com/paper/2205.14620/) (2022)
- [Mask Propagation for Efficient Video Semantic Segmentation](https://mlsanity.com/paper/2310.18954/) (2023)
- [FOLT: Fast Multiple Object Tracking from UAV-captured Videos Based on Optical Flow](https://mlsanity.com/paper/2308.07207/) (2023)
- [Motion-aware Latent Diffusion Models for Video Frame Interpolation](https://mlsanity.com/paper/2404.13534/) (2024)
- [MDFlow: Unsupervised Optical Flow Learning by Reliable Mutual Knowledge Distillation](https://mlsanity.com/paper/2211.06018/) (2022)
- [ATCA: an Arc Trajectory Based Model with Curvature Attention for Video Frame Interpolation](https://mlsanity.com/paper/2208.00856/) (2022)
- [Cross-Consistent Deep Unfolding Network for Adaptive All-In-One Video Restoration](https://mlsanity.com/paper/2309.01627/) (2023)
- [NeuFlow v2: Push High-Efficiency Optical Flow To the Limit](https://mlsanity.com/paper/2408.10161/) (2024)
- [Parallelizing Optical Flow Estimation on an Ultra-Low Power RISC-V Cluster for Nano-UAV Navigation](https://mlsanity.com/paper/2305.13055/) (2023)
- [Dynamic Frame Interpolation in Wavelet Domain](https://mlsanity.com/paper/2309.03508/) (2023)
- [DeepDR: Deep Structure-Aware RGB-D Inpainting for Diminished Reality](https://mlsanity.com/paper/2312.00532/) (2023)
- [UnDIVE: Generalized Underwater Video Enhancement Using Generative Priors](https://mlsanity.com/paper/2411.05886/) (2024)
- [Progressive Motion Context Refine Network for Efficient Video Frame Interpolation](https://mlsanity.com/paper/2211.06024/) (2022)
- [A Multi-Modal Transformer Network for Action Detection](https://mlsanity.com/paper/2305.19624/) (2023)
- [SciFlow: Empowering Lightweight Optical Flow Models with Self-Cleaning Iterations](https://mlsanity.com/paper/2404.08135/) (2024)
- [EgoPolice: A Benchmark for Egocentric Video Understanding in High-Stakes Police Body-Worn Camera Footage](https://mlsanity.com/paper/2607.06468/) (2026)
- [Optical Flow Based Motion Detection for Autonomous Driving](https://mlsanity.com/paper/2203.11693/) (2022)
- [Weakly-Supervised Optical Flow Estimation for Time-of-Flight](https://mlsanity.com/paper/2210.05298/) (2022)
- [Latent Graph Attention for Enhanced Spatial Context](https://mlsanity.com/paper/2307.04149/) (2023)
- [Erasing the Ephemeral: Joint Camera Refinement and Transient Object Removal for Street View Synthesis](https://mlsanity.com/paper/2311.17634/) (2023)
- [A Semantic and Motion-Aware Spatiotemporal Transformer Network for Action Detection](https://mlsanity.com/paper/2405.08204/) (2024)
- [MCDS-VSS: Moving Camera Dynamic Scene Video Semantic Segmentation by Filtering with Self-Supervised Geometry and Motion](https://mlsanity.com/paper/2405.19921/) (2024)
- [360VFI: A Dataset and Benchmark for Omnidirectional Video Frame Interpolation](https://mlsanity.com/paper/2407.14066/) (2024)
- [Trajectory-aware Shifted State Space Models for Online Video Super-Resolution](https://mlsanity.com/paper/2508.10453/) (2025)
- [CompactFlowNet: Efficient Real-time Optical Flow Estimation on Mobile Devices](https://mlsanity.com/paper/2412.13273/) (2024)
- [REMOTE: Real-time Ego-motion Tracking for Various Endoscopes via Multimodal Visual Feature Learning](https://mlsanity.com/paper/2501.18124/) (2025)
- [VelocityNet: Real-Time Crowd Anomaly Detection via Person-Specific Velocity Analysis](https://mlsanity.com/paper/2510.18187/) (2025)
- [Motion-guided sparse correction enables expert-quality point tracking across diverse microscopy regimes](https://mlsanity.com/paper/2605.29220/) (2026)

## Abstract (arXiv)

Dense optical flow estimation plays a key role in many robotic vision tasks. In the past few years, with the advent of deep learning, we have witnessed great progress in optical flow estimation. However, current networks often consist of a large number of parameters and require heavy computation costs, largely hindering its application on low power-consumption devices such as mobile phones. In this paper, we tackle this challenge and design a lightweight model for fast and accurate optical flow prediction. Our proposed FastFlowNet follows the widely-used coarse-to-fine paradigm with following innovations. First, a new head enhanced pooling pyramid (HEPP) feature extractor is employed to intensify high-resolution pyramid features while reducing parameters. Second, we introduce a new center dense dilated correlation (CDDC) layer for constructing compact cost volume that can keep large search radius with reduced computation burden. Third, an efficient shuffle block decoder (SBD) is implanted into each pyramid level to accelerate flow estimation with marginal drops in accuracy. Experiments on both synthetic Sintel data and real-world KITTI datasets demonstrate the effectiveness of the proposed approach, which needs only 1/10 computation of comparable networks to achieve on par accuracy. In particular, FastFlowNet only contains 1.37M parameters; and can execute at 90 FPS (with a single GTX 1080Ti) or 5.7 FPS (embedded Jetson TX2 GPU) on a pair of Sintel images of resolution 1024x436.