Accurate Optical Flow via Direct Cost Volume Processing
Not on the map. Why?
Abstract (arXiv)
We present an optical flow estimation approach that operates on the full four-dimensional cost volume. This direct approach shares the structural benefits of leading stereo matching pipelines, which are known to yield high accuracy. To this day, such approaches have been considered impractical due to the size of the cost volume. We show that the full four-dimensional cost volume can be constructed in a fraction of a second due to its regularity. We then exploit this regularity further by adapting semi-global matching to the four-dimensional setting. This yields a pipeline that achieves significantly higher accuracy than state-of-the-art optical flow methods while being faster than most. Our approach outperforms all published general-purpose optical flow methods on both Sintel and KITTI 2015 benchmarks.
Key facts. Cited by 66 papers in the map (data through 7 Oct 2026). First posted on arXiv 24 Apr 2017.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 25; 2022: 7; 2023: 9; 2024: 16; 2025: 8; 2026: 1.
66 papers.
Sort:
By area:
- 2023CoTracker: It is Better to Track Togethercited by 516
- 2021Learning to Estimate Hidden Motions with Global Motion Aggregationcited by 223
- 2024SpatialTracker: Tracking Any 2D Pixels in 3D Spacecited by 178
- 2024SEA-RAFT: Simple, Efficient, Accurate RAFT for Optical Flowcited by 169
- 2021Track to Detect and Segment: An Online Multi-Object Trackercited by 156
- 2021E-RAFT: Dense Optical Flow from Event Camerascited by 149
- 2021Efficient Regional Memory Network for Video Object Segmentationcited by 86
- 2022CRAFT: Cross-Attentional Flow Transformer for Robust Optical Flowcited by 82
- 2023Siamese Masked Autoencoderscited by 75
- 2021SMURF: Self-Teaching Multi-Frame Unsupervised RAFT with Full-Image Warpingcited by 74
- 2022Global Matching with Overlapping Attention for Optical Flow Estimationcited by 64
- 2024DINO-Tracker: Taming DINO for Self-Supervised Point Tracking in a Single Videocited by 64
38 more
- 2022Deep Equilibrium Optical Flow Estimationcited by 63
- 2024DELTA: Dense Efficient Long-range 3D Tracking for any videocited by 62
- 2023Dense Optical Tracking: Connecting the Dotscited by 58
- 2021High-Resolution Optical Flow from 1D Attention and Correlationcited by 54
- 2025AllTracker: Efficient Dense Point Tracking at High Resolutioncited by 54
- 2021Learning Optical Flow from a Few Matchescited by 53
- 2024TAPTR: Tracking Any Point with Transformers as Detectioncited by 43
- 2021Extreme Rotation Estimation using Dense Correlation Volumescited by 42
- 2025WAFT: Warping-Alone Field Transforms for Optical Flowcited by 32
- 2021Optical Flow Estimation for Spiking Cameracited by 29
- 2021Learning optical flow from still imagescited by 28
- 2024Local Feature Matching Using Deep Learning: A Surveycited by 26
- 2021FCCDN: Feature Constraint Network for VHR Image Change Detectioncited by 25
- 2023DropMAE: Learning Representations via Masked Autoencoders with Spatial-Attention Dropout for Temporal Matching Taskscited by 22
- 2022Spatio-Temporal Deformable Attention Network for Video Deblurringcited by 21
- 2021The Functional Correspondence Problemcited by 20
- 2021Deep Depth from Focus with Differential Focus Volumecited by 20
- 2024TAPTRv2: Attention-based Position Update Improves Tracking Any Pointcited by 18
- 2024Autogenic Language Embedding for Coherent Point Trackingcited by 17
- 2021Overcoming Obstructions via Bandwidth-Limited Multi-Agent Spatial Handshakingcited by 16
- 2023SAMConvex: Fast Discrete Optimization for CT Registration using Self-supervised Anatomical Embedding and Correlation Pyramidcited by 14
- 2021Optical Flow Estimation from a Single Motion-blurred Imagecited by 11
- 2023FlowText: Synthesizing Realistic Scene Text Video with Optical Flow Estimationcited by 10
- 2024Trajectory-aligned Space-time Tokens for Few-shot Action Recognitioncited by 10
- 2021M4Depth: Monocular depth estimation for autonomous vehicles in unseen environmentscited by 9
- 2021FastRIFE: Optimization of Real-Time Intermediate Flow Estimation for Video Frame Interpolationcited by 9
- 2025DELTAv2: Accelerating Dense 3D Trackingcited by 9
- 2024TAPTRv3: Spatial and Temporal Context Foster Robust Tracking of Any Point in Long Videocited by 8
- 2021Enhanced Correlation Matching based Video Frame Interpolationcited by 7
- 2024NeuFlow: Real-time, High-accuracy Optical Flow Estimation on Robots Using Edge Devicescited by 7
- 2021Learning by Distillation: A Self-Supervised Learning Framework for Optical Flow Estimationcited by 6
- 2021Sensor-Guided Optical Flowcited by 4
- 2021Joint Detection of Motion Boundaries and Occlusionscited by 4
- 2023Learning Fine-Grained Features for Pixel-wise Video Correspondencescited by 4
- 2023Learning Residual Elastic Warps for Image Stitching under Dirichlet Boundary Conditioncited by 4
- 2021Target Detection and Segmentation in Circular-Scan Synthetic-Aperture-Sonar Images using Semi-Supervised Convolutional Encoder-Decoderscited by 3
- 2021Normalized Convolution Upsampling for Refined Optical Flow Estimationcited by 3
- 2025ProbDiffFlow: An Efficient Learning-Free Framework for Probabilistic Single-Image Optical Flow Estimationcited by 2
50 most cited of 66 All 66 (JSON)
- 2026Two-Stream Interactive Joint Learning of Scene Parsing and Geometric Vision Tasks14 Feb 2026
- 2025A Stitch in Time: Learning Procedural Workflow via Self-Supervised Plackett-Luce Ranking21 Nov 2025
- 2025Injecting Frame-Event Complementary Fusion into Diffusion for Optical Flow in Challenging Scenes12 Oct 2025
- 2025DELTAv2: Accelerating Dense 3D Tracking2 Aug 2025
- 2025WAFT: Warping-Alone Field Transforms for Optical Flow26 Jun 2025
- 2025AllTracker: Efficient Dense Point Tracking at High Resolution8 Jun 2025
- 2025FDSG: Forecasting Dynamic Scene Graphs2 Jun 2025
- 2025Efficient All-Pairs Correlation Volume Sampling for Optical Flow Estimation22 May 2025
- 2025ProbDiffFlow: An Efficient Learning-Free Framework for Probabilistic Single-Image Optical Flow Estimation16 Mar 2025
- 2024CompactFlowNet: Efficient Real-time Optical Flow Estimation on Mobile Devices17 Dec 2024
- 2024TAPTRv3: Spatial and Temporal Context Foster Robust Tracking of Any Point in Long Video27 Nov 2024
- 2024Cell as Point: One-Stage Framework for Efficient Cell Tracking22 Nov 2024
38 more
- 2024DELTA: Dense Efficient Long-range 3D Tracking for any video31 Oct 2024
- 2024Leveraging Object Priors for Point Tracking9 Sep 2024
- 2024HMAFlow: Learning More Accurate Optical Flow via Hierarchical Motion Field Alignment9 Sep 2024
- 2024Autogenic Language Embedding for Coherent Point Tracking30 Jul 2024
- 2024Trajectory-aligned Space-time Tokens for Few-shot Action Recognition25 Jul 2024
- 2024TAPTRv2: Attention-based Position Update Improves Tracking Any Point23 Jul 2024
- 2024Multi-view Disparity Estimation Using a Novel Gradient Consistency Model27 May 2024
- 2024SEA-RAFT: Simple, Efficient, Accurate RAFT for Optical Flow23 May 2024
- 2024SpatialTracker: Tracking Any 2D Pixels in 3D Space5 Apr 2024
- 2024DINO-Tracker: Taming DINO for Self-Supervised Point Tracking in a Single Video21 Mar 2024
- 2024TAPTR: Tracking Any Point with Transformers as Detection19 Mar 2024
- 2024NeuFlow: Real-time, High-accuracy Optical Flow Estimation on Robots Using Edge Devices15 Mar 2024
- 2024Local Feature Matching Using Deep Learning: A Survey31 Jan 2024
- 2023Dense Optical Tracking: Connecting the Dots1 Dec 2023
- 2023Learning Residual Elastic Warps for Image Stitching under Dirichlet Boundary Condition4 Sep 2023
- 2023The Devil in the Details: Simple and Effective Optical Flow Synthetic Data Generation14 Aug 2023
- 2023Learning Fine-Grained Features for Pixel-wise Video Correspondences6 Aug 2023
- 2023SAMConvex: Fast Discrete Optimization for CT Registration using Self-supervised Anatomical Embedding and Correlation Pyramid19 Jul 2023
- 2023CoTracker: It is Better to Track Together14 Jul 2023
- 2023Siamese Masked Autoencoders23 May 2023
- 2023FlowText: Synthesizing Realistic Scene Text Video with Optical Flow Estimation5 May 2023
- 2023DropMAE: Learning Representations via Masked Autoencoders with Spatial-Attention Dropout for Temporal Matching Tasks2 Apr 2023
- 2022CGCV:Context Guided Correlation Volume for Optical Flow Neural Networks20 Dec 2022
- 2022Spatio-Temporal Deformable Attention Network for Video Deblurring22 Jul 2022
- 2022Complementing Brightness Constancy with Deep Networks for Optical Flow Prediction8 Jul 2022
- 2022FlowNAS: Neural Architecture Search for Optical Flow Estimation4 Jul 2022
- 2022Deep Equilibrium Optical Flow Estimation18 Apr 2022
- 2022CRAFT: Cross-Attentional Flow Transformer for Robust Optical Flow31 Mar 2022
- 2022Global Matching with Overlapping Attention for Optical Flow Estimation21 Mar 2022
- 2021Deep Depth from Focus with Differential Focus Volume3 Dec 2021
- 2021Enhanced Correlation Matching based Video Frame Interpolation17 Nov 2021
- 2021Joint Detection of Motion Boundaries and Occlusions1 Nov 2021
- 2021Optical Flow Estimation for Spiking Camera8 Oct 2021
- 2021Sensor-Guided Optical Flow30 Sep 2021
- 2021The Functional Correspondence Problem2 Sep 2021
- 2021E-RAFT: Dense Optical Flow from Event Cameras24 Aug 2021
- 2021External-Memory Networks for Low-Shot Learning of Targets in Forward-Looking-Sonar Imagery22 Jul 2021
- 2021Overcoming Obstructions via Bandwidth-Limited Multi-Agent Spatial Handshaking1 Jul 2021
50 newest of 66 All 66 (JSON)
- 2023CoTracker: It is Better to Track Togethercited by 516
- 2021Learning to Estimate Hidden Motions with Global Motion Aggregationcited by 223
- 2024SpatialTracker: Tracking Any 2D Pixels in 3D Spacecited by 178
- 2024SEA-RAFT: Simple, Efficient, Accurate RAFT for Optical Flowcited by 169
- 2021Track to Detect and Segment: An Online Multi-Object Trackercited by 156
- 2021E-RAFT: Dense Optical Flow from Event Camerascited by 149
- 2021Efficient Regional Memory Network for Video Object Segmentationcited by 86
- 2022CRAFT: Cross-Attentional Flow Transformer for Robust Optical Flowcited by 82
- 2023Siamese Masked Autoencoderscited by 75
- 2021SMURF: Self-Teaching Multi-Frame Unsupervised RAFT with Full-Image Warpingcited by 74
- 2022Global Matching with Overlapping Attention for Optical Flow Estimationcited by 64
- 2024DINO-Tracker: Taming DINO for Self-Supervised Point Tracking in a Single Videocited by 64
12 most cited of 64 All 64 (JSON)
- 2026Two-Stream Interactive Joint Learning of Scene Parsing and Geometric Vision Tasks14 Feb 2026
- 2025A Stitch in Time: Learning Procedural Workflow via Self-Supervised Plackett-Luce Ranking21 Nov 2025
- 2025DELTAv2: Accelerating Dense 3D Tracking2 Aug 2025
- 2025WAFT: Warping-Alone Field Transforms for Optical Flow26 Jun 2025
- 2025AllTracker: Efficient Dense Point Tracking at High Resolution8 Jun 2025
- 2025FDSG: Forecasting Dynamic Scene Graphs2 Jun 2025
- 2025Efficient All-Pairs Correlation Volume Sampling for Optical Flow Estimation22 May 2025
- 2025ProbDiffFlow: An Efficient Learning-Free Framework for Probabilistic Single-Image Optical Flow Estimation16 Mar 2025
- 2024CompactFlowNet: Efficient Real-time Optical Flow Estimation on Mobile Devices17 Dec 2024
- 2024TAPTRv3: Spatial and Temporal Context Foster Robust Tracking of Any Point in Long Video27 Nov 2024
- 2024Cell as Point: One-Stage Framework for Efficient Cell Tracking22 Nov 2024
- 2024DELTA: Dense Efficient Long-range 3D Tracking for any video31 Oct 2024
12 newest of 64 All 64 (JSON)