# DIP: Deep Inverse Patchmatch for High-Resolution Optical Flow

arXiv: https://arxiv.org/abs/2204.00330 · HTML: https://mlsanity.com/paper/2204.00330/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 59 references that are in the map.
- First posted on arXiv 1 Apr 2022.

## Builds on (most cited first, 28 of 28)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Two-Stream Convolutional Networks for Action Recognition in Videos](https://mlsanity.com/paper/1406.2199/) (2014)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume](https://mlsanity.com/paper/1709.02371/) (2017)
- [A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimation](https://mlsanity.com/paper/1512.02134/) (2015)
- [FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks](https://mlsanity.com/paper/1612.01925/) (2016)
- [Pyramid Stereo Matching Network](https://mlsanity.com/paper/1803.08669/) (2018)
- [GA-Net: Guided Aggregation Net for End-to-end Stereo Matching](https://mlsanity.com/paper/1904.06587/) (2019)
- [Group-wise Correlation Stereo Network](https://mlsanity.com/paper/1903.04025/) (2019)
- [Learning to Estimate Hidden Motions with Global Motion Aggregation](https://mlsanity.com/paper/2104.02409/) (2021)
- [PatchmatchNet: Learned Multi-View Patchmatch Stereo](https://mlsanity.com/paper/2012.01411/) (2020)
- [Super-Convergence: Very Fast Training of Neural Networks Using Large Learning Rates](https://mlsanity.com/paper/1708.07120/) (2017)
- [CFNet: Cascade and Fused Cost Volume for Robust Stereo Matching](https://mlsanity.com/paper/2104.04314/) (2021)
- [MaskFlownet: Asymmetric Feature Matching with Learnable Occlusion Mask](https://mlsanity.com/paper/2003.10955/) (2020)
- [DeepPruner: Learning Efficient Stereo Matching via Differentiable PatchMatch](https://mlsanity.com/paper/1909.05845/) (2019)
- [Domain-invariant Stereo Matching Networks](https://mlsanity.com/paper/1911.13287/) (2019)
- [Models Matter, So Does Training: An Empirical Study of CNNs for Optical Flow Estimation](https://mlsanity.com/paper/1809.05571/) (2018)
- [Hierarchical Discrete Distribution Decomposition for Match Density Estimation](https://mlsanity.com/paper/1812.06264/) (2018)
- [SegFlow: Joint Learning for Video Object Segmentation and Optical Flow](https://mlsanity.com/paper/1709.06750/) (2017)
- [A Lightweight Optical Flow CNN - Revisiting Data Fidelity and Regularization](https://mlsanity.com/paper/1903.07414/) (2019)
- [Fast Optical Flow using Dense Inverse Search](https://mlsanity.com/paper/1603.03590/) (2016)
- [High-Resolution Optical Flow from 1D Attention and Correlation](https://mlsanity.com/paper/2104.13918/) (2021)
- [Learning Optical Flow from a Few Matches](https://mlsanity.com/paper/2104.02166/) (2021)
- [Displacement-Invariant Matching Cost Learning for Accurate Optical Flow Estimation](https://mlsanity.com/paper/2010.14851/) (2020)
- [LiteFlowNet3: Resolving Correspondence Ambiguity for More Accurate Optical Flow Estimation](https://mlsanity.com/paper/2007.09319/) (2020)
- [ScopeFlow: Dynamic Scene Scoping for Optical Flow](https://mlsanity.com/paper/2002.10770/) (2020)
- [End-to-End Learning of Video Super-Resolution with Motion Compensation](https://mlsanity.com/paper/1707.00471/) (2017)

## Built on by (most cited first, 30 of 40)

- [Unifying Flow, Stereo and Depth Estimation](https://mlsanity.com/paper/2211.05783/) (2022)
- [SEA-RAFT: Simple, Efficient, Accurate RAFT for Optical Flow](https://mlsanity.com/paper/2405.14793/) (2024)
- [CroCo v2: Improved Cross-view Completion Pre-training for Stereo Matching and Optical Flow](https://mlsanity.com/paper/2211.10408/) (2022)
- [MemFlow: Optical Flow Estimation and Prediction with Memory](https://mlsanity.com/paper/2404.04808/) (2024)
- [WAFT: Warping-Alone Field Transforms for Optical Flow](https://mlsanity.com/paper/2506.21526/) (2025)
- [Rethinking Optical Flow from Geometric Matching Consistent Perspective](https://mlsanity.com/paper/2303.08384/) (2023)
- [DPFlow: Adaptive Optical Flow Estimation with a Dual-Pyramid Framework](https://mlsanity.com/paper/2503.14880/) (2025)
- [DropMAE: Learning Representations via Masked Autoencoders with Spatial-Attention Dropout for Temporal Matching Tasks](https://mlsanity.com/paper/2304.00571/) (2023)
- [GAFlow: Incorporating Gaussian Attention into Optical Flow](https://mlsanity.com/paper/2309.16217/) (2023)
- [Boosting Object Representation Learning via Motion and Object Continuity](https://mlsanity.com/paper/2211.09771/) (2022)
- [AnyFlow: Arbitrary Scale Optical Flow with Implicit Neural Representation](https://mlsanity.com/paper/2303.16493/) (2023)
- [CCMR: High Resolution Optical Flow Estimation via Coarse-to-Fine Context-Guided Motion Reasoning](https://mlsanity.com/paper/2311.02661/) (2023)
- [FlowSeek: Optical Flow Made Easier with Depth Foundation Models and Motion Bases](https://mlsanity.com/paper/2509.05297/) (2025)
- [Memory-Efficient Optical Flow via Radius-Distribution Orthogonal Cost Volume](https://mlsanity.com/paper/2312.03790/) (2023)
- [Flow-Anything: Learning Real-World Optical Flow Estimation from Large-Scale Single-view Images](https://mlsanity.com/paper/2506.07740/) (2025)
- [DIFT: Dynamic Iterative Field Transforms for Memory Efficient Optical Flow](https://mlsanity.com/paper/2306.05691/) (2023)
- [MEMFOF: High-Resolution Training for Memory-Efficient Multi-Frame Optical Flow Estimation](https://mlsanity.com/paper/2506.23151/) (2025)
- [Integrative Feature and Cost Aggregation with Transformers for Dense Correspondence](https://mlsanity.com/paper/2209.08742/) (2022)
- [Hybrid Cost Volume for Memory-Efficient Optical Flow](https://mlsanity.com/paper/2409.04243/) (2024)
- [EDCFlow: Exploring Temporally Dense Difference Maps for Event-based Optical Flow Estimation](https://mlsanity.com/paper/2506.03512/) (2025)
- [ProMotion: Prototypes As Motion Learners](https://mlsanity.com/paper/2406.04999/) (2024)
- [NeuFlow v2: Push High-Efficiency Optical Flow To the Limit](https://mlsanity.com/paper/2408.10161/) (2024)
- [MegaFlow: Zero-Shot Large Displacement Optical Flow](https://mlsanity.com/paper/2603.25739/) (2026)
- [Prototypical Transformer as Unified Motion Learners](https://mlsanity.com/paper/2406.01559/) (2024)
- [Open-DDVM: A Reproduction and Extension of Diffusion Model for Optical Flow Estimation](https://mlsanity.com/paper/2312.01746/) (2023)
- [Online Dense Point Tracking with Streaming Memory](https://mlsanity.com/paper/2503.06471/) (2025)
- [MambaFlow: A Mamba-Centric Architecture for End-to-End Optical Flow Estimation](https://mlsanity.com/paper/2503.07046/) (2025)
- [Image-Coupled Volume Propagation for Stereo Matching](https://mlsanity.com/paper/2301.00695/) (2022)
- [Efficient All-Pairs Correlation Volume Sampling for Optical Flow Estimation](https://mlsanity.com/paper/2505.16942/) (2025)
- [Towards Reliable Sea Ice Drift Estimation in the Arctic Deep Learning Optical Flow on RADARSAT-2](https://mlsanity.com/paper/2510.26653/) (2025)

## Abstract (arXiv)

Recently, the dense correlation volume method achieves state-of-the-art performance in optical flow. However, the correlation volume computation requires a lot of memory, which makes prediction difficult on high-resolution images. In this paper, we propose a novel Patchmatch-based framework to work on high-resolution optical flow estimation. Specifically, we introduce the first end-to-end Patchmatch based deep learning optical flow. It can get high-precision results with lower memory benefiting from propagation and local search of Patchmatch. Furthermore, a new inverse propagation is proposed to decouple the complex operations of propagation, which can significantly reduce calculations in multiple iterations. At the time of submission, our method ranks first on all the metrics on the popular KITTI2015 benchmark, and ranks second on EPE on the Sintel clean benchmark among published optical flow methods. Experiment shows our method has a strong cross-dataset generalization ability that the F1-all achieves 13.73%, reducing 21% from the best published result 17.4% on KITTI2015. What's more, our method shows a good details preserving result on the high-resolution dataset DAVIS and consumes 2x less memory than RAFT.