# Back to Basics: Unsupervised Learning of Optical Flow via Brightness Constancy and Motion Smoothness

arXiv: https://arxiv.org/abs/1608.05842 · HTML: https://mlsanity.com/paper/1608.05842/

## Key facts

- Cited by 101 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Aug 2016.

## Built on by (most cited first, 30 of 101)

- [PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Tracking](https://mlsanity.com/paper/2307.15055/) (2023)
- [CroCo v2: Improved Cross-view Completion Pre-training for Stereo Matching and Optical Flow](https://mlsanity.com/paper/2211.10408/) (2022)
- [BootsTAP: Bootstrapped Training for Tracking-Any-Point](https://mlsanity.com/paper/2402.00847/) (2024)
- [SMURF: Self-Teaching Multi-Frame Unsupervised RAFT with Full-Image Warping](https://mlsanity.com/paper/2105.07014/) (2021)
- [MC-JEPA: A Joint-Embedding Predictive Architecture for Self-Supervised Learning of Motion and Content Features](https://mlsanity.com/paper/2307.12698/) (2023)
- [Robust Cross-Modal Representation Learning with Progressive Self-Distillation](https://mlsanity.com/paper/2204.04588/) (2022)
- [Representation Learning via Global Temporal Alignment and Cycle-Consistency](https://mlsanity.com/paper/2105.05217/) (2021)
- [Warp Consistency for Unsupervised Learning of Dense Correspondences](https://mlsanity.com/paper/2104.03308/) (2021)
- [Drop-DTW: Aligning Common Signal Between Sequences While Dropping Outliers](https://mlsanity.com/paper/2108.11996/) (2021)
- [Learning Pixel Trajectories with Multiscale Contrastive Random Walks](https://mlsanity.com/paper/2201.08379/) (2022)
- [Optical flow estimation from event-based cameras and spiking neural networks](https://mlsanity.com/paper/2302.06492/) (2023)
- [Disentangling Architecture and Training for Optical Flow](https://mlsanity.com/paper/2203.10712/) (2022)
- [Learning Optical Flow, Depth, and Scene Flow without Real-World Labels](https://mlsanity.com/paper/2203.15089/) (2022)
- [Cross-View Completion Models are Zero-shot Correspondence Estimators](https://mlsanity.com/paper/2412.09072/) (2024)
- [Optical Flow Estimation for Spiking Camera](https://mlsanity.com/paper/2110.03916/) (2021)
- [Learning optical flow from still images](https://mlsanity.com/paper/2104.03965/) (2021)
- [Fusion-FlowNet: Energy-Efficient Optical Flow Estimation using Sensor Fusion and Deep Fused Spiking-Analog Network Architectures](https://mlsanity.com/paper/2103.10592/) (2021)
- [Motion-prior Contrast Maximization for Dense Continuous-Time Motion Estimation](https://mlsanity.com/paper/2407.10802/) (2024)
- [RealFlow: EM-based Realistic Optical Flow Dataset Generation from Videos](https://mlsanity.com/paper/2207.11075/) (2022)
- [Adaptive-SpikeNet: Event-based Optical Flow Estimation using Spiking Neural Networks with Learnable Neuronal Dynamics](https://mlsanity.com/paper/2209.11741/) (2022)
- [GAFlow: Incorporating Gaussian Attention into Optical Flow](https://mlsanity.com/paper/2309.16217/) (2023)
- [Comparing Correspondences: Video Prediction with Correspondence-wise Losses](https://mlsanity.com/paper/2104.09498/) (2021)
- [GyroFlow: Gyroscope-Guided Unsupervised Optical Flow Learning](https://mlsanity.com/paper/2103.13725/) (2021)
- [Bring Event into RGB and LiDAR: Hierarchical Visual-Motion Fusion for Scene Flow](https://mlsanity.com/paper/2403.07432/) (2024)
- [ChiTransformer:Towards Reliable Stereo from Cues](https://mlsanity.com/paper/2203.04554/) (2022)
- [Optical Flow Training under Limited Label Budget via Active Learning](https://mlsanity.com/paper/2203.05053/) (2022)
- [3DG-STFM: 3D Geometric Guided Student-Teacher Feature Matching](https://mlsanity.com/paper/2207.02375/) (2022)
- [Spatial-then-Temporal Self-Supervised Learning for Video Correspondence](https://mlsanity.com/paper/2209.07778/) (2022)
- [Refining Pre-Trained Motion Models](https://mlsanity.com/paper/2401.00850/) (2024)
- [MOL: Joint Estimation of Micro-Expression, Optical Flow, and Landmark via Transformer-Graph-Style Convolution](https://mlsanity.com/paper/2506.14511/) (2025)

## Abstract (arXiv)

Recently, convolutional networks (convnets) have proven useful for predicting optical flow. Much of this success is predicated on the availability of large datasets that require expensive and involved data acquisition and laborious la- beling. To bypass these challenges, we propose an unsuper- vised approach (i.e., without leveraging groundtruth flow) to train a convnet end-to-end for predicting optical flow be- tween two images. We use a loss function that combines a data term that measures photometric constancy over time with a spatial term that models the expected variation of flow across the image. Together these losses form a proxy measure for losses based on the groundtruth flow. Empiri- cally, we show that a strong convnet baseline trained with the proposed unsupervised approach outperforms the same network trained with supervision on the KITTI dataset.