# DIFT: Dynamic Iterative Field Transforms for Memory Efficient Optical Flow

arXiv: https://arxiv.org/abs/2306.05691 · HTML: https://mlsanity.com/paper/2306.05691/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 28 references that are in the map.
- First posted on arXiv 9 Jun 2023.

## Builds on (most cited first, 21 of 21)

- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation](https://mlsanity.com/paper/1512.02134/) (2015)
- [FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks](https://mlsanity.com/paper/1612.01925/) (2016)
- [A Unifying Review of Deep and Shallow Anomaly Detection](https://mlsanity.com/paper/2009.11732/) (2020)
- [Optical Flow Estimation using a Spatial Pyramid Network](https://mlsanity.com/paper/1611.00850/) (2016)
- [Virtual KITTI 2](https://mlsanity.com/paper/2001.10773/) (2020)
- [GMFlow: Learning Optical Flow via Global Matching](https://mlsanity.com/paper/2111.13680/) (2021)
- [FlowFormer: A Transformer Architecture for Optical Flow](https://mlsanity.com/paper/2203.16194/) (2022)
- [Iterative Residual Refinement for Joint Optical Flow and Occlusion Estimation](https://mlsanity.com/paper/1904.05290/) (2019)
- [AutoFlow: Learning a Better Training Set for Optical Flow](https://mlsanity.com/paper/2104.14544/) (2021)
- [High-Resolution Optical Flow from 1D Attention and Correlation](https://mlsanity.com/paper/2104.13918/) (2021)
- [DIP: Deep Inverse Patchmatch for High-Resolution Optical Flow](https://mlsanity.com/paper/2204.00330/) (2022)
- [Open Category Detection with PAC Guarantees](https://mlsanity.com/paper/1808.00529/) (2018)
- [Neural Network Quantization with AI Model Efficiency Toolkit (AIMET)](https://mlsanity.com/paper/2201.08442/) (2022)
- [Imposing Consistency for Optical Flow Estimation](https://mlsanity.com/paper/2204.07262/) (2022)
- [Towards Understanding Adversarial Robustness of Optical Flow Networks](https://mlsanity.com/paper/2103.16255/) (2021)
- [A Learned Stereo Depth System for Robotic Manipulation in Homes](https://mlsanity.com/paper/2109.11644/) (2021)
- [DistractFlow: Improving Optical Flow Estimation via Realistic Distractions and Pseudo-Labeling](https://mlsanity.com/paper/2303.14078/) (2023)
- [Oracle Analysis of Representations for Deep Open Set Detection](https://mlsanity.com/paper/2209.11350/) (2022)

## Built on by (most cited first, 7 of 7)

- [SEA-RAFT: Simple, Efficient, Accurate RAFT for Optical Flow](https://mlsanity.com/paper/2405.14793/) (2024)
- [MAMo: Leveraging Memory and Attention for Monocular Video Depth Estimation](https://mlsanity.com/paper/2307.14336/) (2023)
- [CostFilter-AD: Enhancing Anomaly Detection through Matching Cost Filtering](https://mlsanity.com/paper/2505.01476/) (2025)
- [SciFlow: Empowering Lightweight Optical Flow Models with Self-Cleaning Iterations](https://mlsanity.com/paper/2404.08135/) (2024)
- [VHS: High-Resolution Iterative Stereo Matching with Visual Hull Priors](https://mlsanity.com/paper/2406.02552/) (2024)
- [STATIC : Surface Temporal Affine for TIme Consistency in Video Monocular Depth Estimation](https://mlsanity.com/paper/2412.01090/) (2024)
- [Efficient All-Pairs Correlation Volume Sampling for Optical Flow Estimation](https://mlsanity.com/paper/2505.16942/) (2025)

## Abstract (arXiv)

Recent advancements in neural network-based optical flow estimation often come with prohibitively high computational and memory requirements, presenting challenges in their model adaptation for mobile and low-power use cases. In this paper, we introduce a lightweight low-latency and memory-efficient model, Dynamic Iterative Field Transforms (DIFT), for optical flow estimation feasible for edge applications such as mobile, XR, micro UAVs, robotics and cameras. DIFT follows an iterative refinement framework leveraging variable resolution of cost volumes for correspondence estimation. We propose a memory efficient solution for cost volume processing to reduce peak memory. Also, we present a novel dynamic coarse-to-fine cost volume processing during various stages of refinement to avoid multiple levels of cost volumes. We demonstrate first real-time cost-volume based optical flow DL architecture on Snapdragon 8 Gen 1 HTP efficient mobile AI accelerator with 32 inf/sec and 5.89 EPE (endpoint error) on KITTI with manageable accuracy-performance tradeoffs.