mlsanityData to 14 Oct 2026

Learning optical flow from still images

arXiv 2104.03965 ↗First posted 8 Apr 2021cs.CVcited by 28

Not on the map. Why?

Abstract (arXiv)

This paper deals with the scarcity of data for training optical flow networks, highlighting the limitations of existing sources such as labeled synthetic datasets or unlabeled real videos. Specifically, we introduce a framework to generate accurate ground-truth optical flow annotations quickly and in large amounts from any readily available single real picture. Given an image, we use an off-the-shelf monocular depth estimation network to build a plausible point cloud for the observed scene. Then, we virtually move the camera in the reconstructed environment with known motion vectors and rotation angles, allowing us to synthesize both a novel view and the corresponding optical flow field connecting each pixel in the input image to the one in the new frame. When trained with our data, state-of-the-art optical flow networks achieve superior generalization to unseen real data compared to the same models trained either on annotated synthetic datasets or unlabeled videos, and better specialization if combined with synthetic images.

Key facts. Cited by 28 papers in the map (data through 14 Oct 2026). Builds on 47 of its 68 references that are in the map. First posted on arXiv 8 Apr 2021.

Builds on

47 of 68 references.

References · 47

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2017Mask R-CNNcited by 7,277
  4. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,415
  5. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,495
  6. 2017CyCADA: Cycle-Consistent Adversarial Domain Adaptationcited by 1,082
  7. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,059
  8. 2017PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volumecited by 968
  9. 2015A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimationcited by 881
  10. 2016Unsupervised Monocular Depth Estimation with Left-Right Consistencycited by 820
  11. 2016FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networkscited by 784
  12. 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 782
35 more
  1. 2018MegaDepth: Learning Single-View Depth Prediction from Internet Photoscited by 737
  2. 2018Deep Ordinal Regression Network for Monocular Depth Estimationcited by 723
  3. 20193D Packing for Self-Supervised Monocular Depth Estimationcited by 439
  4. 2016Deeper Depth Prediction with Fully Convolutional Residual Networkscited by 403
  5. 2017Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolationcited by 326
  6. 2020Softmax Splatting for Video Frame Interpolationcited by 279
  7. 2020Single-View View Synthesis with Multiplane Imagescited by 263
  8. 2019DeepView: View Synthesis with Learned Gradient Descentcited by 253
  9. 2019Depth-Aware Video Frame Interpolationcited by 234
  10. 2020Free View Synthesiscited by 231
  11. 2018Stereo Magnification: Learning View Synthesis using Multiplane Imagescited by 211
  12. 20203D Photography using Context-aware Layered Depth Inpaintingcited by 206
  13. 2017UnFlow: Unsupervised Learning of Optical Flow with a Bidirectional Census Losscited by 202
  14. 2017Image to Image Translation for Domain Adaptationcited by 171
  15. 2020Novel View Synthesis of Dynamic Scenes with Globally Coherent Depths from a Monocular Cameracited by 166
  16. 2019SelFlow: Self-Supervised Learning of Optical Flowcited by 128
  17. 20193D Ken Burns Effect from a Single Imagecited by 128
  18. 2019Self-Supervised Monocular Depth Hintscited by 127
  19. 2020What Matters in Unsupervised Optical Flowcited by 124
  20. 2020Learning by Analogy: Reliable Supervision from Transformations for Unsupervised Optical Flow Estimationcited by 120
  21. 2019Domain-invariant Stereo Matching Networkscited by 109
  22. 2016Back to Basics: Unsupervised Learning of Optical Flow via Brightness Constancy and Motion Smoothnesscited by 101
  23. 2018Models Matter, So Does Training: An Empirical Study of CNNs for Optical Flow Estimationcited by 94
  24. 2020Unsupervised Domain Adaptation in Semantic Segmentation: a Reviewcited by 88
  25. 2018Real-time self-adaptive deep stereocited by 80
  26. 2018Structured Attention Guided Convolutional Neural Fields for Monocular Depth Estimationcited by 78
  27. 2019A Lightweight Optical Flow CNN - Revisiting Data Fidelity and Regularizationcited by 76
  28. 2017Accurate Optical Flow via Direct Cost Volume Processingcited by 66
  29. 2019Learning monocular depth estimation infusing traditional stereo knowledgecited by 64
  30. 2019DDFlow: Learning Optical Flow with Unlabeled Data Distillationcited by 63
  31. 2020Learning Stereo from Single Imagescited by 45
  32. 2020Distilled Semantics for Comprehensive Scene Understanding from Videoscited by 39
  33. 2020LiteFlowNet3: Resolving Correspondence Ambiguity for More Accurate Optical Flow Estimationcited by 37
  34. 2016Full Flow: Optical Flow Estimation By Global Optimization over Regular Gridscited by 34
  35. 2019Learning Across Tasks and Domainscited by 16

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 4; 2022: 8; 2023: 6; 2024: 1; 2025: 7; 2026: 2.

28 papers.

Sort:

By area:

  1. 2022Deep Equilibrium Optical Flow Estimationcited by 64
  2. 2022Single-View View Synthesis in the Wild with Learned Adaptive Multiplane Imagescited by 59
  3. 2023NeRF-Supervised Deep Stereocited by 46
  4. 2022IFOR: Iterative Flow Minimization for Robotic Object Rearrangementcited by 42
  5. 2023Rethinking Optical Flow from Geometric Matching Consistent Perspectivecited by 33
  6. 2022RealFlow: EM-based Realistic Optical Flow Dataset Generation from Videoscited by 22
  7. 2025Unaligned RGB Guided Hyperspectral Image Super-Resolution with Spatial-Spectral Concordancecited by 19
  8. 2025Pixel Motion as Universal Representation for Robot Controlcited by 15
  9. 2022Robust Reflection Removal with Flash-only Cues in the Wildcited by 14
  10. 20213D Hierarchical Refinement and Augmentation for Unsupervised Learning of Depth and Pose from Monocular Videocited by 11
  11. 2022Self-supervised AutoFlowcited by 10
  12. 2025FlowSeek: Optical Flow Made Easier with Depth Foundation Models and Motion Basescited by 10
16 more
  1. 2023Unsupervised Cumulative Domain Adaptation for Foggy Scene Optical Flowcited by 9
  2. 2021360deg Optical Flow using Tangent Imagescited by 7
  3. 2023MPI-Flow: Learning Realistic Optical Flow with Multiplane Imagescited by 7
  4. 2026Future Optical Flow Prediction Improves Robot Control & Video Generationcited by 7
  5. 2025VRM: Knowledge Distillation via Virtual Relation Matchingcited by 6
  6. 2021Sensor-Guided Optical Flowcited by 4
  7. 2024Adverse Weather Optical Flow: Cumulative Homogeneous-Heterogeneous Adaptationcited by 4
  8. 2025Online Dense Point Tracking with Streaming Memorycited by 3
  9. 2025ProbDiffFlow: An Efficient Learning-Free Framework for Probabilistic Single-Image Optical Flow Estimationcited by 2
  10. 2025StableMTL: Repurposing Latent Diffusion Models for Multi-Task Learning from Partially Annotated Synthetic Datasetscited by 2
  11. 2021Learning multiplane images from single views with self-supervisioncited by 1
  12. 2022Weakly-Supervised Optical Flow Estimation for Time-of-Flightcited by 1
  13. 2022SizeGAN: Improving Size Representation in Clothing Catalogscited by 1
  14. 2023The Devil in the Details: Simple and Effective Optical Flow Synthetic Data Generationcited by 0
  15. 2023Skin the sheep not only once: Reusing Various Depth Datasets to Drive the Learning of Optical Flowcited by 0
  16. 2026FlowIt: Global Matching via Hierarchical Transformers and Optimal Transport for Optical Flowcited by 0