mlsanityData to 14 Oct 2026

Optical Flow Estimation using a Spatial Pyramid Network

arXiv 1611.00850 ↗First posted 3 Nov 2016cs.CVcited by 424

Not on the map. Why?

Abstract (arXiv)

We learn to compute optical flow by combining a classical spatial-pyramid formulation with deep learning. This estimates large motions in a coarse-to-fine approach by warping one image of a pair at each pyramid level by the current flow estimate and computing an update to the flow. Instead of the standard minimization of an objective function at each pyramid level, we train one deep network per level to compute the flow update. Unlike the recent FlowNet approach, the networks do not need to deal with large motions; these are dealt with by the pyramid. This has several advantages. First, our Spatial Pyramid Network (SPyNet) is much simpler and 96% smaller than FlowNet in terms of model parameters. This makes it more efficient and appropriate for embedded applications. Second, since the flow at each pyramid level is small (< 1 pixel), a convolutional approach applied to pairs of warped images is appropriate. Third, unlike FlowNet, the learned convolution filters appear similar to classical spatio-temporal filters, giving insight into the method and how to improve it. Our results are more accurate than FlowNet on most standard benchmarks, suggesting a new direction of combining classical flow methods with deep learning.

Key facts. Cited by 424 papers in the map (data through 14 Oct 2026). First posted on arXiv 3 Nov 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 68; 2022: 95; 2023: 86; 2024: 84; 2025: 57; 2026: 34.

424 papers.

Sort:

By area:

  1. 2024Grounding Image Matching in 3D with MASt3Rcited by 983
  2. 2021BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignmentcited by 342
  3. 2021GMFlow: Learning Optical Flow via Global Matchingcited by 327
  4. 2023TAPIR: Tracking Any Point with per-frame Initialization and temporal Refinementcited by 287
  5. 2022FlowFormer: A Transformer Architecture for Optical Flowcited by 275
  6. 2022Unifying Flow, Stereo and Depth Estimationcited by 275
  7. 2022TAP-Vid: A Benchmark for Tracking Any Point in a Videocited by 263
  8. 2021Deep Contextual Video Compressioncited by 221
  9. 2022VRT: A Video Restoration Transformercited by 220
  10. 2024Motion-I2V: Consistent and Controllable Image-to-Video Generation with Explicit Motion Modelingcited by 211
  11. 2023Spring: A High-Resolution High-Detail Dataset and Benchmark for Scene Flow, Optical Flow and Stereocited by 172
  12. 2022Recurrent Video Restoration Transformer with Guided Deformable Attentioncited by 170
38 more
  1. 2022Hybrid Spatial-Temporal Entropy Modelling for Neural Video Compressioncited by 128
  2. 2021Video Super-Resolution Transformercited by 121
  3. 2021Temporal Context Mining for Learned Video Compressioncited by 117
  4. 2023FlowFormer++: Masked Cost Volume Autoencoding for Pretraining Optical Flow Estimationcited by 114
  5. 2022Towards An End-to-End Framework for Flow-Guided Video Inpaintingcited by 113
  6. 2022IFRNet: Intermediate Feature Refine Network for Efficient Frame Interpolationcited by 109
  7. 2021AutoFlow: Learning a Better Training Set for Optical Flowcited by 102
  8. 2021Asymmetric Bilateral Motion Estimation for Video Frame Interpolationcited by 101
  9. 2025DriveSuprim: Towards Precise Trajectory Selection for End-to-End Planningcited by 94
  10. 2023VideoControlNet: A Motion-Guided Video-to-Video Translation Framework by Using Diffusion Model with ControlNetcited by 83
  11. 2022CRAFT: Cross-Attentional Flow Transformer for Robust Optical Flowcited by 82
  12. 2023VideoFlow: Exploiting Temporal Cues for Multi-frame Optical Flow Estimationcited by 82
  13. 2021Rethinking Self-supervised Correspondence Learning: A Video Frame-level Similarity Perspectivecited by 81
  14. 2022VIDM: Video Implicit Diffusion Modelscited by 78
  15. 2021SMURF: Self-Teaching Multi-Frame Unsupervised RAFT with Full-Image Warpingcited by 74
  16. 2022Rethinking Alignment in Video Super-Resolution Transformerscited by 71
  17. 2022A Simple Baseline for Video Restoration with Grouped Spatial-temporal Shiftcited by 68
  18. 2024Ctrl-Adapter: An Efficient and Versatile Framework for Adapting Diverse Controls to Any Diffusion Modelcited by 66
  19. 2024DELTA: Dense Efficient Long-range 3D Tracking for any videocited by 65
  20. 2022Deep Equilibrium Optical Flow Estimationcited by 64
  21. 2022Learning Trajectory-Aware Transformer for Video Super-Resolutioncited by 61
  22. 2023Dense Optical Tracking: Connecting the Dotscited by 61
  23. 2023Motion-Guided Latent Diffusion for Temporally Consistent Real-world Video Super-resolutioncited by 59
  24. 2023MC-JEPA: A Joint-Embedding Predictive Architecture for Self-Supervised Learning of Motion and Content Featurescited by 57
  25. 2021CamLiFlow: Bidirectional Camera-LiDAR Fusion for Joint Optical Flow and Scene Flow Estimationcited by 56
  26. 2022Coarse-to-fine Deep Video Coding with Hyperprior-guided Mode Predictioncited by 55
  27. 2024MemFlow: Optical Flow Estimation and Prediction with Memorycited by 55
  28. 2023A Dynamic Multi-Scale Voxel Flow Network for Video Predictioncited by 53
  29. 2022Wireless Deep Video Semantic Transmissioncited by 50
  30. 2021MGSampler: An Explainable Sampling Strategy for Video Action Recognitioncited by 49
  31. 2022CANF-VC: Conditional Augmented Normalizing Flows for Video Compressioncited by 46
  32. 2021The Emergence of Objectness: Learning Zero-Shot Segmentation from Videoscited by 45
  33. 2022Flow-Guided Sparse Transformer for Video Deblurringcited by 43
  34. 2021Perceptual Learned Video Compression with Recurrent Conditional GANcited by 41
  35. 2024USTC-TD: A Test Dataset and Benchmark for Image and Video Coding in 2020scited by 37
  36. 2023Enhancing Video Super-Resolution via Implicit Resampling-based Alignmentcited by 35
  37. 2025WAFT: Warping-Alone Field Transforms for Optical Flowcited by 34
  38. 2021Deep Learning for UAV-based Object Detection and Tracking: A Surveycited by 33

50 most cited of 424 All 424 (JSON)