mlsanityData to 7 Oct 2026

FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks

arXiv 1612.01925 ↗First posted 6 Dec 2016cs.CVcited by 780

Not on the map. Why?

Abstract (arXiv)

The FlowNet demonstrated that optical flow estimation can be cast as a learning problem. However, the state of the art with regard to the quality of the flow has still been defined by traditional methods. Particularly on small displacements and real-world data, FlowNet cannot compete with variational methods. In this paper, we advance the concept of end-to-end learning of optical flow and make it work really well. The large improvements in quality and speed are caused by three major contributions: first, we focus on the training data and show that the schedule of presenting data during training is very important. Second, we develop a stacked architecture that includes warping of the second image with intermediate optical flow. Third, we elaborate on small displacements by introducing a sub-network specializing on small motions. FlowNet 2.0 is only marginally slower than the original FlowNet but decreases the estimation error by more than 50%. It performs on par with state-of-the-art methods, while running at interactive frame rates. Moreover, we present faster variants that allow optical flow computation at up to 140fps with accuracy matching the original FlowNet.

Key facts. Cited by 780 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Dec 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 224; 2022: 180; 2023: 153; 2024: 122; 2025: 73; 2026: 28.

780 papers.

Sort:

By area:

  1. 2023CoTracker: It is Better to Track Togethercited by 516
  2. 2021GMFlow: Learning Optical Flow via Global Matchingcited by 324
  3. 2023TAPIR: Tracking Any Point with per-frame Initialization and temporal Refinementcited by 281
  4. 2023PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Trackingcited by 274
  5. 2022FlowFormer: A Transformer Architecture for Optical Flowcited by 273
  6. 2022Unifying Flow, Stereo and Depth Estimationcited by 270
  7. 2022TAP-Vid: A Benchmark for Tracking Any Point in a Videocited by 258
  8. 2024Shape of Motion: 4D Reconstruction from a Single Videocited by 247
  9. 2021Keeping Your Eye on the Ball: Trajectory Attention in Video Transformerscited by 224
  10. 2021Learning to Estimate Hidden Motions with Global Motion Aggregationcited by 223
  11. 2023Drag Your GAN: Interactive Point-based Manipulation on the Generative Image Manifoldcited by 218
  12. 2022Particle Video Revisited: Tracking Through Occlusions Using Point Trajectoriescited by 215
38 more
  1. 2024Motion-I2V: Consistent and Controllable Image-to-Video Generation with Explicit Motion Modelingcited by 209
  2. 2021Pixel-Perfect Structure-from-Motion with Featuremetric Refinementcited by 199
  3. 2021NerfingMVS: Guided Optimization of Neural Radiance Fields for Indoor Multi-view Stereocited by 191
  4. 2023Tracking Everything Everywhere All at Oncecited by 190
  5. 2024SpatialTracker: Tracking Any 2D Pixels in 3D Spacecited by 178
  6. 2022ASpanFormer: Detector-Free Image Matching with Adaptive Span Transformercited by 170
  7. 2023Spring: A High-Resolution High-Detail Dataset and Benchmark for Scene Flow, Optical Flow and Stereocited by 170
  8. 2024SEA-RAFT: Simple, Efficient, Accurate RAFT for Optical Flowcited by 169
  9. 2023Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Drivingcited by 168
  10. 2021TimeLens: Event-based Video Frame Interpolationcited by 150
  11. 2021A Hybrid Video Anomaly Detection Framework via Memory-Augmented Flow Reconstruction and Flow-Guided Frame Predictioncited by 129
  12. 2022DRAMA: Joint Risk Localization and Captioning in Drivingcited by 125
  13. 2023GP-VTON: Towards General Purpose Virtual Try-on via Collaborative Local-Flow Global-Parsing Learningcited by 120
  14. 2021Learning to Track with Object Permanencecited by 117
  15. 2023The Surprising Effectiveness of Diffusion Models for Optical Flow and Monocular Depth Estimationcited by 116
  16. 2023FlowFormer++: Masked Cost Volume Autoencoding for Pretraining Optical Flow Estimationcited by 114
  17. 2021Learning Accurate Dense Correspondences and When to Trust Themcited by 113
  18. 2021Stylizing 3D Scene via Implicit Representation and HyperNetworkcited by 103
  19. 2021AutoFlow: Learning a Better Training Set for Optical Flowcited by 102
  20. 2021Asymmetric Bilateral Motion Estimation for Video Frame Interpolationcited by 101
  21. 2022FlowBot3D: Learning 3D Articulation Flow to Manipulate Articulated Objectscited by 91
  22. 2025DriveSuprim: Towards Precise Trajectory Selection for End-to-End Planningcited by 91
  23. 2022ParticleSfM: Exploiting Dense Point Trajectories for Localizing Moving Cameras in the Wildcited by 90
  24. 2022Style-Based Global Appearance Flow for Virtual Try-Oncited by 88
  25. 2021Efficient Regional Memory Network for Video Object Segmentationcited by 86
  26. 2022CRAFT: Cross-Attentional Flow Transformer for Robust Optical Flowcited by 82
  27. 2023VideoFlow: Exploiting Temporal Cues for Multi-frame Optical Flow Estimationcited by 82
  28. 2021Rethinking Self-supervised Correspondence Learning: A Video Frame-level Similarity Perspectivecited by 81
  29. 2021PDC-Net+: Enhanced Probabilistic Dense Correspondence Networkcited by 80
  30. 2022Unsupervised Misaligned Infrared and Visible Image Fusion via Cross-Modality Image Generation and Registrationcited by 80
  31. 2021Full-Duplex Strategy for Video Object Segmentationcited by 78
  32. 2021CATs: Cost Aggregation Transformers for Visual Correspondencecited by 77
  33. 2023Siamese Masked Autoencoderscited by 75
  34. 2021SMURF: Self-Teaching Multi-Frame Unsupervised RAFT with Full-Image Warpingcited by 74
  35. 2021STEP: Segmenting and Tracking Every Pixelcited by 72
  36. 2022Learning to Relight Portrait Images via a Virtual Light Stage and Synthetic-to-Real Adaptationcited by 67
  37. 2022Physical Adversarial Attack meets Computer Vision: A Decade Surveycited by 66
  38. 2022Global Matching with Overlapping Attention for Optical Flow Estimationcited by 64

50 most cited of 780 All 780 (JSON)