mlsanityData to 14 Oct 2026

Back to Basics: Unsupervised Learning of Optical Flow via Brightness Constancy and Motion Smoothness

arXiv 1608.05842 ↗First posted 20 Aug 2016cs.CVcited by 101

Not on the map. Why?

Abstract (arXiv)

Recently, convolutional networks (convnets) have proven useful for predicting optical flow. Much of this success is predicated on the availability of large datasets that require expensive and involved data acquisition and laborious la- beling. To bypass these challenges, we propose an unsuper- vised approach (i.e., without leveraging groundtruth flow) to train a convnet end-to-end for predicting optical flow be- tween two images. We use a loss function that combines a data term that measures photometric constancy over time with a spatial term that models the expected variation of flow across the image. Together these losses form a proxy measure for losses based on the groundtruth flow. Empiri- cally, we show that a strong convnet baseline trained with the proposed unsupervised approach outperforms the same network trained with supervision on the KITTI dataset.

Key facts. Cited by 101 papers in the map (data through 14 Oct 2026). First posted on arXiv 20 Aug 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 25; 2022: 33; 2023: 20; 2024: 10; 2025: 9; 2026: 4.

101 papers.

Sort:

By area:

  1. 2023PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Trackingcited by 276
  2. 2022CroCo v2: Improved Cross-view Completion Pre-training for Stereo Matching and Optical Flowcited by 166
  3. 2024BootsTAP: Bootstrapped Training for Tracking-Any-Pointcited by 120
  4. 2021SMURF: Self-Teaching Multi-Frame Unsupervised RAFT with Full-Image Warpingcited by 74
  5. 2023MC-JEPA: A Joint-Embedding Predictive Architecture for Self-Supervised Learning of Motion and Content Featurescited by 57
  6. 2022Robust Cross-Modal Representation Learning with Progressive Self-Distillationcited by 55
  7. 2021Representation Learning via Global Temporal Alignment and Cycle-Consistencycited by 44
  8. 2021Warp Consistency for Unsupervised Learning of Dense Correspondencescited by 42
  9. 2021Drop-DTW: Aligning Common Signal Between Sequences While Dropping Outlierscited by 39
  10. 2022Learning Pixel Trajectories with Multiscale Contrastive Random Walkscited by 36
  11. 2023Optical flow estimation from event-based cameras and spiking neural networkscited by 33
  12. 2022Disentangling Architecture and Training for Optical Flowcited by 32
38 more
  1. 2022Learning Optical Flow, Depth, and Scene Flow without Real-World Labelscited by 31
  2. 2024Cross-View Completion Models are Zero-shot Correspondence Estimatorscited by 30
  3. 2021Optical Flow Estimation for Spiking Cameracited by 29
  4. 2021Learning optical flow from still imagescited by 28
  5. 2021Fusion-FlowNet: Energy-Efficient Optical Flow Estimation using Sensor Fusion and Deep Fused Spiking-Analog Network Architecturescited by 25
  6. 2024Motion-prior Contrast Maximization for Dense Continuous-Time Motion Estimationcited by 24
  7. 2022RealFlow: EM-based Realistic Optical Flow Dataset Generation from Videoscited by 22
  8. 2022Adaptive-SpikeNet: Event-based Optical Flow Estimation using Spiking Neural Networks with Learnable Neuronal Dynamicscited by 19
  9. 2023GAFlow: Incorporating Gaussian Attention into Optical Flowcited by 18
  10. 2021Comparing Correspondences: Video Prediction with Correspondence-wise Lossescited by 16
  11. 2021GyroFlow: Gyroscope-Guided Unsupervised Optical Flow Learningcited by 15
  12. 2024Bring Event into RGB and LiDAR: Hierarchical Visual-Motion Fusion for Scene Flowcited by 15
  13. 2022ChiTransformer:Towards Reliable Stereo from Cuescited by 14
  14. 2022Optical Flow Training under Limited Label Budget via Active Learningcited by 14
  15. 20223DG-STFM: 3D Geometric Guided Student-Teacher Feature Matchingcited by 14
  16. 2022Spatial-then-Temporal Self-Supervised Learning for Video Correspondencecited by 13
  17. 2024Refining Pre-Trained Motion Modelscited by 12
  18. 2025MOL: Joint Estimation of Micro-Expression, Optical Flow, and Landmark via Transformer-Graph-Style Convolutioncited by 12
  19. 2022Unsupervised Flow-Aligned Sequence-to-Sequence Learning for Video Restorationcited by 11
  20. 2022MDFlow: Unsupervised Optical Flow Learning by Reliable Mutual Knowledge Distillationcited by 10
  21. 2022Self-supervised AutoFlowcited by 10
  22. 2023GyroFlow+: Gyroscope-Guided Unsupervised Deep Homography and Optical Flow Learningcited by 10
  23. 2023Self-Supervised Motion Magnification by Backpropagating Through Optical Flowcited by 10
  24. 2023Fully Convolutional Slice-to-Volume Reconstruction for Single-Stack MRIcited by 10
  25. 2024Self-Supervised Any-Point Tracking by Contrastive Random Walkscited by 10
  26. 2025Scene-Centric Unsupervised Panoptic Segmentationcited by 10
  27. 2023SemARFlow: Injecting Semantics into Unsupervised Optical Flow Estimation for Autonomous Drivingcited by 9
  28. 2023Unsupervised Cumulative Domain Adaptation for Foggy Scene Optical Flowcited by 9
  29. 2021ASFlow: Unsupervised Optical Flow Learning with Adaptive Pyramid Samplingcited by 8
  30. 2022Stochastic Video Prediction with Structure and Motioncited by 8
  31. 2022Self-Supervised Super-Resolution for Multi-Exposure Push-Frame Satellitescited by 8
  32. 2022SuperWarp: Supervised Learning and Warping on U-Net for Invariant Subvoxel-Precise Registrationcited by 8
  33. 2024UnSAMFlow: Unsupervised Optical Flow Guided by Segment Anything Modelcited by 8
  34. 2023SplatFlow: Learning Multi-frame Optical Flow via Splattingcited by 7
  35. 2021LIFE: Lighting Invariant Flow Estimationcited by 6
  36. 2021Track, Check, Repeat: An EM Approach to Unsupervised Trackingcited by 6
  37. 2021Learning by Distillation: A Self-Supervised Learning Framework for Optical Flow Estimationcited by 6
  38. 2023Doduo: Learning Dense Visual Correspondence from Unsupervised Semantic-Aware Flowcited by 6

50 most cited of 101 All 101 (JSON)