# MEMFOF: High-Resolution Training for Memory-Efficient Multi-Frame Optical Flow Estimation

arXiv: https://arxiv.org/abs/2506.23151 · HTML: https://mlsanity.com/paper/2506.23151/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 41 references that are in the map.
- First posted on arXiv 29 Jun 2025.

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks](https://mlsanity.com/paper/1612.01925/) (2016)
- [GMFlow: Learning Optical Flow via Global Matching](https://mlsanity.com/paper/2111.13680/) (2021)
- [BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond](https://mlsanity.com/paper/2012.02181/) (2020)
- [FlowFormer: A Transformer Architecture for Optical Flow](https://mlsanity.com/paper/2203.16194/) (2022)
- [Learning to Estimate Hidden Motions with Global Motion Aggregation](https://mlsanity.com/paper/2104.02409/) (2021)
- [Real-Time Intermediate Flow Estimation for Video Frame Interpolation](https://mlsanity.com/paper/2011.06294/) (2020)
- [SEA-RAFT: Simple, Efficient, Accurate RAFT for Optical Flow](https://mlsanity.com/paper/2405.14793/) (2024)
- [Spring: A High-Resolution High-Detail Dataset and Benchmark for Scene Flow, Optical Flow and Stereo](https://mlsanity.com/paper/2303.01943/) (2023)
- [Playing for Benchmarks](https://mlsanity.com/paper/1709.07322/) (2017)
- [CroCo v2: Improved Cross-view Completion Pre-training for Stereo Matching and Optical Flow](https://mlsanity.com/paper/2211.10408/) (2022)
- [Quadratic video interpolation](https://mlsanity.com/paper/1911.00627/) (2019)
- [The Surprising Effectiveness of Diffusion Models for Optical Flow and Monocular Depth Estimation](https://mlsanity.com/paper/2306.01923/) (2023)
- [FlowFormer++: Masked Cost Volume Autoencoding for Pretraining Optical Flow Estimation](https://mlsanity.com/paper/2303.01237/) (2023)
- [Optical Flow Guided Feature: A Fast and Robust Motion Representation for Video Action Recognition](https://mlsanity.com/paper/1711.11152/) (2017)
- [VideoFlow: Exploiting Temporal Cues for Multi-frame Optical Flow Estimation](https://mlsanity.com/paper/2303.08340/) (2023)
- [MemFlow: Optical Flow Estimation and Prediction with Memory](https://mlsanity.com/paper/2404.04808/) (2024)
- [High-Resolution Optical Flow from 1D Attention and Correlation](https://mlsanity.com/paper/2104.13918/) (2021)
- [Learning Optical Flow from a Few Matches](https://mlsanity.com/paper/2104.02166/) (2021)
- [Representation Flow for Action Recognition](https://mlsanity.com/paper/1810.01455/) (2018)
- [DIP: Deep Inverse Patchmatch for High-Resolution Optical Flow](https://mlsanity.com/paper/2204.00330/) (2022)
- [A Fusion Approach for Multi-Frame Optical Flow Estimation](https://mlsanity.com/paper/1810.10066/) (2018)
- [Face Deblurring using Dual Camera Fusion on Mobile Phones](https://mlsanity.com/paper/2207.11617/) (2022)
- [StreamFlow: Streamlined Multi-Frame Optical Flow Estimation for Video Sequences](https://mlsanity.com/paper/2311.17099/) (2023)
- [Memory-Efficient Optical Flow via Radius-Distribution Orthogonal Cost Volume](https://mlsanity.com/paper/2312.03790/) (2023)
- [Hybrid Cost Volume for Memory-Efficient Optical Flow](https://mlsanity.com/paper/2409.04243/) (2024)

## Built on by (most cited first, 7 of 7)

- [EgoGrasp: World-Space Hand-Object Interaction Estimation from Egocentric Videos](https://mlsanity.com/paper/2601.01050/) (2026)
- [MegaFlow: Zero-Shot Large Displacement Optical Flow](https://mlsanity.com/paper/2603.25739/) (2026)
- [Towards Reliable Sea Ice Drift Estimation in the Arctic Deep Learning Optical Flow on RADARSAT-2](https://mlsanity.com/paper/2510.26653/) (2025)
- [Gaussians on Fire: High-Frequency Reconstruction of Flames](https://mlsanity.com/paper/2511.22459/) (2025)
- [FreeTimeGS++: Secrets of Dynamic Gaussian Splatting and Their Principles](https://mlsanity.com/paper/2605.03337/) (2026)
- [FreeFlow: A Bias-free Hierarchical Transformer for Optical Flow Estimation](https://mlsanity.com/paper/2609.11486/) (2026)
- [MemReg: Streaming Outdoor LiDAR Point Cloud Registration with Hybrid Memory Buffers](https://mlsanity.com/paper/2610.12624/) (2026)

## Abstract (arXiv)

Recent advances in optical flow estimation have prioritized accuracy at the cost of growing GPU memory consumption, particularly for high-resolution (FullHD) inputs. We introduce MEMFOF, a memory-efficient multi-frame optical flow method that identifies a favorable trade-off between multi-frame estimation and GPU memory usage. Notably, MEMFOF requires only 2.09 GB of GPU memory at runtime for 1080p inputs, and 28.5 GB during training, which uniquely positions our method to be trained at native 1080p without the need for cropping or downsampling. We systematically revisit design choices from RAFT-like architectures, integrating reduced correlation volumes and high-resolution training protocols alongside multi-frame estimation, to achieve state-of-the-art performance across multiple benchmarks while substantially reducing memory overhead. Our method outperforms more resource-intensive alternatives in both accuracy and runtime efficiency, validating its robustness for flow estimation at high resolutions. At the time of submission, our method ranks first on the Spring benchmark with a 1-pixel (1px) outlier rate of 3.289, leads Sintel (clean) with an endpoint error (EPE) of 0.963, and achieves the best Fl-all error on KITTI-2015 at 2.94%. The code is available at https://github.com/msu-video-group/memfof.