# SciFlow: Empowering Lightweight Optical Flow Models with Self-Cleaning Iterations

arXiv: https://arxiv.org/abs/2404.08135 · HTML: https://mlsanity.com/paper/2404.08135/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 49 references that are in the map.
- First posted on arXiv 11 Apr 2024.

## Builds on (most cited first, 30 of 42)

- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation](https://mlsanity.com/paper/1512.02134/) (2015)
- [FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks](https://mlsanity.com/paper/1612.01925/) (2016)
- [Optical Flow Estimation using a Spatial Pyramid Network](https://mlsanity.com/paper/1611.00850/) (2016)
- [DVC: An End-to-end Deep Video Compression Framework](https://mlsanity.com/paper/1812.00101/) (2018)
- [FlowFormer: A Transformer Architecture for Optical Flow](https://mlsanity.com/paper/2203.16194/) (2022)
- [GRU-ODE-Bayes: Continuous modeling of sporadically-observed time series](https://mlsanity.com/paper/1905.12374/) (2019)
- [Learning to Estimate Hidden Motions with Global Motion Aggregation](https://mlsanity.com/paper/2104.02409/) (2021)
- [Video Compression through Image Interpolation](https://mlsanity.com/paper/1804.06919/) (2018)
- [MaskFlownet: Asymmetric Feature Matching with Learnable Occlusion Mask](https://mlsanity.com/paper/2003.10955/) (2020)
- [FlowFormer++: Masked Cost Volume Autoencoding for Pretraining Optical Flow Estimation](https://mlsanity.com/paper/2303.01237/) (2023)
- [Learning Accurate Dense Correspondences and When to Trust Them](https://mlsanity.com/paper/2101.01710/) (2021)
- [IFRNet: Intermediate Feature Refine Network for Efficient Frame Interpolation](https://mlsanity.com/paper/2205.14620/) (2022)
- [Hierarchical Discrete Distribution Decomposition for Match Density Estimation](https://mlsanity.com/paper/1812.06264/) (2018)
- [DeepTAM: Deep Tracking and Mapping](https://mlsanity.com/paper/1808.01900/) (2018)
- [Lightweight Probabilistic Deep Networks](https://mlsanity.com/paper/1805.11327/) (2018)
- [A Lightweight Optical Flow CNN - Revisiting Data Fidelity and Regularization](https://mlsanity.com/paper/1903.07414/) (2019)
- [High-Resolution Optical Flow from 1D Attention and Correlation](https://mlsanity.com/paper/2104.13918/) (2021)
- [Learning Optical Flow from a Few Matches](https://mlsanity.com/paper/2104.02166/) (2021)
- [Displacement-Invariant Matching Cost Learning for Accurate Optical Flow Estimation](https://mlsanity.com/paper/2010.14851/) (2020)
- [LiteFlowNet3: Resolving Correspondence Ambiguity for More Accurate Optical Flow Estimation](https://mlsanity.com/paper/2007.09319/) (2020)
- [Motion Feature Network: Fixed Motion Filter for Action Recognition](https://mlsanity.com/paper/1807.10037/) (2018)
- [FastFlowNet: A Lightweight Network for Fast Optical Flow Estimation](https://mlsanity.com/paper/2103.04524/) (2021)
- [Neural Network Quantization with AI Model Efficiency Toolkit (AIMET)](https://mlsanity.com/paper/2201.08442/) (2022)
- [Imposing Consistency for Optical Flow Estimation](https://mlsanity.com/paper/2204.07262/) (2022)

## Built on by (most cited first, 2 of 2)

- [Improving Optical Flow and Stereo Depth Estimation by Leveraging Uncertainty-Based Learning Difficulties](https://mlsanity.com/paper/2506.00324/) (2025)
- [SciFlow: Semantic Cross Interference for Self-Supervised Optical Flow Domain Generalization](https://mlsanity.com/paper/2606.29004/) (2026)

## Abstract (arXiv)

Optical flow estimation is crucial to a variety of vision tasks. Despite substantial recent advancements, achieving real-time on-device optical flow estimation remains a complex challenge. First, an optical flow model must be sufficiently lightweight to meet computation and memory constraints to ensure real-time performance on devices. Second, the necessity for real-time on-device operation imposes constraints that weaken the model's capacity to adequately handle ambiguities in flow estimation, thereby intensifying the difficulty of preserving flow accuracy. This paper introduces two synergistic techniques, Self-Cleaning Iteration (SCI) and Regression Focal Loss (RFL), designed to enhance the capabilities of optical flow models, with a focus on addressing optical flow regression ambiguities. These techniques prove particularly effective in mitigating error propagation, a prevalent issue in optical flow models that employ iterative refinement. Notably, these techniques add negligible to zero overhead in model parameters and inference latency, thereby preserving real-time on-device efficiency. The effectiveness of our proposed SCI and RFL techniques, collectively referred to as SciFlow for brevity, is demonstrated across two distinct lightweight optical flow model architectures in our experiments. Remarkably, SciFlow enables substantial reduction in error metrics (EPE and Fl-all) over the baseline models by up to 6.3% and 10.5% for in-domain scenarios and by up to 6.2% and 13.5% for cross-domain scenarios on the Sintel and KITTI 2015 datasets, respectively.