A Fusion Approach for Multi-Frame Optical Flow Estimation
Not on the map. Why?
Abstract (arXiv)
To date, top-performing optical flow estimation methods only take pairs of consecutive frames into account. While elegant and appealing, the idea of using more than two frames has not yet produced state-of-the-art results. We present a simple, yet effective fusion approach for multi-frame optical flow that benefits from longer-term temporal cues. Our method first warps the optical flow from previous frames to the current, thereby yielding multiple plausible estimates. It then fuses the complementary information carried by these estimates into a new optical flow field. At the time of writing, our method ranks first among published results in the MPI Sintel and KITTI 2015 benchmarks. Our models will be available on https://github.com/NVlabs/PWC-Net.
Key facts. Cited by 37 papers in the map (data through 7 Oct 2026). First posted on arXiv 23 Oct 2018.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 5; 2022: 7; 2023: 13; 2024: 7; 2025: 4; 2026: 1.
37 papers.
Sort:
- 2023CoTracker: It is Better to Track Togethercited by 516
- 2024Shape of Motion: 4D Reconstruction from a Single Videocited by 247
- 2022Particle Video Revisited: Tracking Through Occlusions Using Point Trajectoriescited by 215
- 2023Tracking Everything Everywhere All at Oncecited by 190
- 2024SpatialTracker: Tracking Any 2D Pixels in 3D Spacecited by 178
- 2023AMT: All-Pairs Multi-Field Transforms for Efficient Frame Interpolationcited by 177
- 2023VideoFlow: Exploiting Temporal Cues for Multi-frame Optical Flow Estimationcited by 82
- 2023MFT: Long-Term Tracking of Every Pixelcited by 61
- 2024MemFlow: Optical Flow Estimation and Prediction with Memorycited by 55
- 2021Spatio-Temporal Recurrent Networks for Event-Based Optical Flow Estimationcited by 50
- 2022Dense Continuous-Time Optical Flow from Events and Framescited by 45
- 2021Self-Supervised Multi-Frame Monocular Scene Flowcited by 29
25 more
- 2023AccFlow: Backward Accumulation for Long-Range Optical Flowcited by 19
- 2022Optical Flow Training under Limited Label Budget via Active Learningcited by 14
- 2022M-FUSE: Multi-frame Fusion for Scene Flow Estimationcited by 14
- 2023StreamFlow: Streamlined Multi-Frame Optical Flow Estimation for Video Sequencescited by 12
- 2025L4P: Towards Unified Low-Level 4D Vision Perceptioncited by 12
- 2023SemARFlow: Injecting Semantics into Unsupervised Optical Flow Estimation for Autonomous Drivingcited by 9
- 2023Context-PIPs: Persistent Independent Particles Demands Spatial Context Featurescited by 9
- 2022Deep 360^circ Optical Flow Estimation Based on Multi-Projection Fusioncited by 8
- 2023SplatFlow: Learning Multi-frame Optical Flow via Splattingcited by 7
- 2021Learning by Distillation: A Self-Supervised Learning Framework for Optical Flow Estimationcited by 6
- 2025MEMFOF: High-Resolution Training for Memory-Efficient Multi-Frame Optical Flow Estimationcited by 6
- 2021Detail Preserving Residual Feature Pyramid Modules for Optical Flowcited by 4
- 2022Emerging Threats in Deep Learning-Based Autonomous Driving: A Comprehensive Surveycited by 3
- 2024Self-Supervised Multi-Frame Neural Scene Flowcited by 3
- 2024GG-SSMs: Graph-Generating State Space Modelscited by 3
- 2023SSTM: Spatiotemporal Recurrent Transformers for Multi-frame Optical Flow Estimationcited by 2
- 2025Perturbed State Space Feature Encoders for Optical Flow with Event Camerascited by 2
- 2021Prediction-assistant Frame Super-Resolution for Video Streamingcited by 1
- 2023UFD-PRiME: Unsupervised Joint Learning of Optical Flow and Stereo Depth through Pixel-Level Rigid Motion Estimationcited by 1
- 2024CathFlow: Self-Supervised Segmentation of Catheters in Interventional Ultrasound Using Optical Flow and Transformerscited by 1
- 2022Meta-Interpolation: Time-Arbitrary Frame Interpolation via Dual Meta-Learningcited by 0
- 2023MaskFlow: Object-Aware Motion Estimationcited by 0
- 2024CompactFlowNet: Efficient Real-time Optical Flow Estimation on Mobile Devicescited by 0
- 2025Gaussians on Fire: High-Frequency Reconstruction of Flamescited by 0
- 2026RAFT-MSF++: Temporal Geometry-Motion Feature Fusion for Self-Supervised Monocular Scene Flowcited by 0
- 2026RAFT-MSF++: Temporal Geometry-Motion Feature Fusion for Self-Supervised Monocular Scene Flow21 Apr 2026
- 2025Gaussians on Fire: High-Frequency Reconstruction of Flames27 Nov 2025
- 2025MEMFOF: High-Resolution Training for Memory-Efficient Multi-Frame Optical Flow Estimation29 Jun 2025
- 2025Perturbed State Space Feature Encoders for Optical Flow with Event Cameras14 Apr 2025
- 2025L4P: Towards Unified Low-Level 4D Vision Perception18 Feb 2025
- 2024CompactFlowNet: Efficient Real-time Optical Flow Estimation on Mobile Devices17 Dec 2024
- 2024GG-SSMs: Graph-Generating State Space Models17 Dec 2024
- 2024Shape of Motion: 4D Reconstruction from a Single Video18 Jul 2024
- 2024MemFlow: Optical Flow Estimation and Prediction with Memory7 Apr 2024
- 2024SpatialTracker: Tracking Any 2D Pixels in 3D Space5 Apr 2024
- 2024Self-Supervised Multi-Frame Neural Scene Flow24 Mar 2024
- 2024CathFlow: Self-Supervised Segmentation of Catheters in Interventional Ultrasound Using Optical Flow and Transformers21 Mar 2024
25 more
- 2023StreamFlow: Streamlined Multi-Frame Optical Flow Estimation for Video Sequences28 Nov 2023
- 2023MaskFlow: Object-Aware Motion Estimation21 Nov 2023
- 2023UFD-PRiME: Unsupervised Joint Learning of Optical Flow and Stereo Depth through Pixel-Level Rigid Motion Estimation7 Oct 2023
- 2023AccFlow: Backward Accumulation for Long-Range Optical Flow25 Aug 2023
- 2023CoTracker: It is Better to Track Together14 Jul 2023
- 2023SplatFlow: Learning Multi-frame Optical Flow via Splatting15 Jun 2023
- 2023Tracking Everything Everywhere All at Once8 Jun 2023
- 2023Context-PIPs: Persistent Independent Particles Demands Spatial Context Features3 Jun 2023
- 2023MFT: Long-Term Tracking of Every Pixel22 May 2023
- 2023SSTM: Spatiotemporal Recurrent Transformers for Multi-frame Optical Flow Estimation26 Apr 2023
- 2023AMT: All-Pairs Multi-Field Transforms for Efficient Frame Interpolation19 Apr 2023
- 2023VideoFlow: Exploiting Temporal Cues for Multi-frame Optical Flow Estimation15 Mar 2023
- 2023SemARFlow: Injecting Semantics into Unsupervised Optical Flow Estimation for Autonomous Driving10 Mar 2023
- 2022Emerging Threats in Deep Learning-Based Autonomous Driving: A Comprehensive Survey19 Oct 2022
- 2022Deep 360^circ Optical Flow Estimation Based on Multi-Projection Fusion27 Jul 2022
- 2022Meta-Interpolation: Time-Arbitrary Frame Interpolation via Dual Meta-Learning27 Jul 2022
- 2022M-FUSE: Multi-frame Fusion for Scene Flow Estimation12 Jul 2022
- 2022Particle Video Revisited: Tracking Through Occlusions Using Point Trajectories8 Apr 2022
- 2022Dense Continuous-Time Optical Flow from Events and Frames25 Mar 2022
- 2022Optical Flow Training under Limited Label Budget via Active Learning9 Mar 2022
- 2021Spatio-Temporal Recurrent Networks for Event-Based Optical Flow Estimation10 Sep 2021
- 2021Detail Preserving Residual Feature Pyramid Modules for Optical Flow23 Jul 2021
- 2021Learning by Distillation: A Self-Supervised Learning Framework for Optical Flow Estimation8 Jun 2021
- 2021Self-Supervised Multi-Frame Monocular Scene Flow5 May 2021
- 2021Prediction-assistant Frame Super-Resolution for Video Streaming17 Mar 2021
- 2023CoTracker: It is Better to Track Togethercited by 516
- 2024Shape of Motion: 4D Reconstruction from a Single Videocited by 247
- 2022Particle Video Revisited: Tracking Through Occlusions Using Point Trajectoriescited by 215
- 2023Tracking Everything Everywhere All at Oncecited by 190
- 2024SpatialTracker: Tracking Any 2D Pixels in 3D Spacecited by 178
- 2023AMT: All-Pairs Multi-Field Transforms for Efficient Frame Interpolationcited by 177
- 2023VideoFlow: Exploiting Temporal Cues for Multi-frame Optical Flow Estimationcited by 82
- 2023MFT: Long-Term Tracking of Every Pixelcited by 61
- 2024MemFlow: Optical Flow Estimation and Prediction with Memorycited by 55
- 2021Spatio-Temporal Recurrent Networks for Event-Based Optical Flow Estimationcited by 50
- 2022Dense Continuous-Time Optical Flow from Events and Framescited by 45
- 2021Self-Supervised Multi-Frame Monocular Scene Flowcited by 29
12 most cited of 37 All 37 (JSON)
- 2026RAFT-MSF++: Temporal Geometry-Motion Feature Fusion for Self-Supervised Monocular Scene Flow21 Apr 2026
- 2025Gaussians on Fire: High-Frequency Reconstruction of Flames27 Nov 2025
- 2025MEMFOF: High-Resolution Training for Memory-Efficient Multi-Frame Optical Flow Estimation29 Jun 2025
- 2025Perturbed State Space Feature Encoders for Optical Flow with Event Cameras14 Apr 2025
- 2025L4P: Towards Unified Low-Level 4D Vision Perception18 Feb 2025
- 2024CompactFlowNet: Efficient Real-time Optical Flow Estimation on Mobile Devices17 Dec 2024
- 2024GG-SSMs: Graph-Generating State Space Models17 Dec 2024
- 2024Shape of Motion: 4D Reconstruction from a Single Video18 Jul 2024
- 2024MemFlow: Optical Flow Estimation and Prediction with Memory7 Apr 2024
- 2024SpatialTracker: Tracking Any 2D Pixels in 3D Space5 Apr 2024
- 2024Self-Supervised Multi-Frame Neural Scene Flow24 Mar 2024
- 2024CathFlow: Self-Supervised Segmentation of Catheters in Interventional Ultrasound Using Optical Flow and Transformers21 Mar 2024
12 newest of 37 All 37 (JSON)