Occlusion Aware Unsupervised Learning of Optical Flow
Not on the map. Why?
Abstract (arXiv)
It has been recently shown that a convolutional neural network can learn optical flow estimation with unsupervised learning. However, the performance of the unsupervised methods still has a relatively large gap compared to its supervised counterpart. Occlusion and large motion are some of the major factors that limit the current unsupervised learning of optical flow methods. In this work we introduce a new method which models occlusion explicitly and a new warping way that facilitates the learning of large motion. Our method shows promising results on Flying Chairs, MPI-Sintel and KITTI benchmark datasets. Especially on KITTI dataset where abundant unlabeled samples exist, our unsupervised method outperforms its counterpart trained with supervised learning.
Key facts. Cited by 101 papers in the map (data through 14 Oct 2026). First posted on arXiv 16 Nov 2017.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 35; 2022: 26; 2023: 13; 2024: 12; 2025: 8; 2026: 7.
101 papers.
Sort:
By area:
- 2021Learning to Estimate Hidden Motions with Global Motion Aggregationcited by 224
- 2022Simple Unsupervised Object-Centric Learning for Complex and Naturalistic Videoscited by 139
- 2024CoTracker3: Simpler and Better Point Tracking by Pseudo-Labelling Real Videoscited by 128
- 2021XVFI: eXtreme Video Frame Interpolationcited by 126
- 2024BootsTAP: Bootstrapped Training for Tracking-Any-Pointcited by 120
- 2021SMURF: Self-Teaching Multi-Frame Unsupervised RAFT with Full-Image Warpingcited by 74
- 2021Self-Supervised Monocular Depth and Ego-Motion Estimation in Endoscopy: Appearance Flow to the Rescuecited by 72
- 2021Learning Monocular Depth in Dynamic Scenes via Instance-Aware Projection Consistencycited by 58
- 2021Self-Supervised Pillar Motion Learning for Autonomous Drivingcited by 44
- 2021Warp Consistency for Unsupervised Learning of Dense Correspondencescited by 42
- 2022Bi-PointFlowNet: Bidirectional Learning for Point Cloud Based Scene Flow Estimationcited by 38
- 2022Learning Pixel Trajectories with Multiscale Contrastive Random Walkscited by 36
38 more
- 2021Instance-wise Occlusion and Depth Orders in Natural Scenescited by 31
- 2024MVPGS: Excavating Multi-view Priors for Gaussian Splatting from Sparse Input Viewscited by 29
- 2021Attention meets Geometry: Geometry Guided Spatial-Temporal Attention for Consistent Self-Supervised Monocular Depth Estimationcited by 27
- 2025Event-Based Video Frame Interpolation With Cross-Modal Asymmetric Bidirectional Motion Fieldscited by 25
- 2022Imposing Consistency for Optical Flow Estimationcited by 24
- 2022RealFlow: EM-based Realistic Optical Flow Dataset Generation from Videoscited by 22
- 2021Generative Transformer for Accurate and Reliable Salient Object Detectioncited by 18
- 2021VOLDOR: Visual Odometry from Log-logistic Dense Optical flow Residualscited by 17
- 2021Comparing Correspondences: Video Prediction with Correspondence-wise Lossescited by 16
- 2021Overcoming Obstructions via Bandwidth-Limited Multi-Agent Spatial Handshakingcited by 16
- 2021GyroFlow: Gyroscope-Guided Unsupervised Optical Flow Learningcited by 15
- 2021Learning Co-segmentation by Segment Swapping for Retrieval and Discoverycited by 15
- 2022Context-Aware Video Reconstruction for Rolling Shutter Camerascited by 15
- 2022Unsupervised Deformable Image Registration with Absent Correspondences in Pre-operative and Post-Recurrence Brain Tumor MRI Scanscited by 15
- 2022Optical Flow Training under Limited Label Budget via Active Learningcited by 14
- 2022Position-Aware Relation Learning for RGB-Thermal Salient Object Detectioncited by 12
- 2021Occlusion Guided Self-supervised Scene Flow Estimation on 3D Point Cloudscited by 11
- 2022Unsupervised Flow-Aligned Sequence-to-Sequence Learning for Video Restorationcited by 11
- 2022MDFlow: Unsupervised Optical Flow Learning by Reliable Mutual Knowledge Distillationcited by 10
- 2022Self-supervised AutoFlowcited by 10
- 2023GyroFlow+: Gyroscope-Guided Unsupervised Deep Homography and Optical Flow Learningcited by 10
- 2023Self-Supervised Motion Magnification by Backpropagating Through Optical Flowcited by 10
- 2024Self-Supervised Any-Point Tracking by Contrastive Random Walkscited by 10
- 2023SemARFlow: Injecting Semantics into Unsupervised Optical Flow Estimation for Autonomous Drivingcited by 9
- 2025AceVFI: A Comprehensive Survey of Advances in Video Frame Interpolationcited by 9
- 2021ASFlow: Unsupervised Optical Flow Learning with Adaptive Pyramid Samplingcited by 8
- 2021Energy-Based Generative Cooperative Saliency Predictioncited by 8
- 2022SuperWarp: Supervised Learning and Warping on U-Net for Invariant Subvoxel-Precise Registrationcited by 8
- 2022Deep 360^circ Optical Flow Estimation Based on Multi-Projection Fusioncited by 8
- 2023Video Task Decathlon: Unifying Image and Video Tasks in Autonomous Drivingcited by 8
- 2024UnSAMFlow: Unsupervised Optical Flow Guided by Segment Anything Modelcited by 8
- 2022Unsupervised Learning of 3D Scene Flow from Monocular Cameracited by 7
- 2023SplatFlow: Learning Multi-frame Optical Flow via Splattingcited by 7
- 2021Learning by Distillation: A Self-Supervised Learning Framework for Optical Flow Estimationcited by 6
- 2022Rolling Shutter Inversion: Bring Rolling Shutter Images to High Framerate Global Shutter Videocited by 6
- 2023Doduo: Learning Dense Visual Correspondence from Unsupervised Semantic-Aware Flowcited by 6
- 2021ORStereo: Occlusion-Aware Recurrent Stereo Matching for 4K-Resolution Imagescited by 5
- 2021There and Back Again: Self-supervised Multispectral Correspondence Estimationcited by 5
50 most cited of 101 All 101 (JSON)
- 2026Triangular Consistency as a Universal Constraint for Learning Optical Flow18 Jun 2026
- 2026Improved monocular depth prediction using distance transform over pre-semantic contours with self-supervised neural networks8 May 2026
- 2026Rebalancing gradient to improve self-supervised co-training of depth, odometry and optical flow predictions8 May 2026
- 2026RAFT-MSF++: Temporal Geometry-Motion Feature Fusion for Self-Supervised Monocular Scene Flow21 Apr 2026
- 2026U2Flow: Uncertainty-Aware Unsupervised Optical Flow Estimation11 Apr 2026
- 2026GenOpticalFlow: A Generative Approach to Unsupervised Optical Flow Learning23 Mar 2026
- 2026Taming Real-World Space-Time Video Super-Resolution with One-Step Diffusion28 Jan 2026
- 2025Visual Autoregressive Modelling for Monocular Depth Estimation27 Dec 2025
- 2025Dynamic Gaussian Splatting from Defocused and Motion-blurred Monocular Videos12 Oct 2025
- 2025EndoMUST: Monocular Depth Estimation for Robotic Endoscopy via End-to-end Multi-step Self-supervised Training19 Jun 2025
- 2025JointSplat: Probabilistic Joint Flow-Depth Optimization for Sparse-View Gaussian Splatting4 Jun 2025
- 2025AceVFI: A Comprehensive Survey of Advances in Video Frame Interpolation1 Jun 2025
38 more
- 2025Joint Flow And Feature Refinement Using Attention For Video Restoration22 May 2025
- 2025UnIRe: Unsupervised Instance Decomposition for Dynamic Urban Scene Reconstruction1 Apr 2025
- 2025Event-Based Video Frame Interpolation With Cross-Modal Asymmetric Bidirectional Motion Fields19 Feb 2025
- 2024CoTracker3: Simpler and Better Point Tracking by Pseudo-Labelling Real Videos15 Oct 2024
- 2024Self-Assessed Generation: Trustworthy Label Generation for Optical Flow and Stereo Matching in Real-world14 Oct 2024
- 2024Pseudo-Stereo Inputs: A Solution to the Occlusion Challenge in Self-Supervised Stereo Matching3 Oct 2024
- 2024Adverse Weather Optical Flow: Cumulative Homogeneous-Heterogeneous Adaptation25 Sep 2024
- 2024Self-Supervised Any-Point Tracking by Contrastive Random Walks24 Sep 2024
- 2024MVPGS: Excavating Multi-view Priors for Gaussian Splatting from Sparse Input Views22 Sep 2024
- 2024Consistency Regularisation for Unsupervised Domain Adaptation in Monocular Depth Estimation27 May 2024
- 2024Rethink Predicting the Optical Flow with the Kinetics Perspective21 May 2024
- 2024UnSAMFlow: Unsupervised Optical Flow Guided by Segment Anything Model4 May 2024
- 2024Ada-Tracker: Soft Tissue Tracking via Inter-Frame and Adaptive-Template Matching11 Mar 2024
- 2024BootsTAP: Bootstrapped Training for Tracking-Any-Point1 Feb 2024
- 2024Key-point Guided Deformable Image Manipulation Using Diffusion Model16 Jan 2024
- 2023Self-Supervised Motion Magnification by Backpropagating Through Optical Flow28 Nov 2023
- 2023MaskFlow: Object-Aware Motion Estimation21 Nov 2023
- 2023UFD-PRiME: Unsupervised Joint Learning of Optical Flow and Stereo Depth through Pixel-Level Rigid Motion Estimation7 Oct 2023
- 2023Doduo: Learning Dense Visual Correspondence from Unsupervised Semantic-Aware Flow26 Sep 2023
- 2023Video Task Decathlon: Unifying Image and Video Tasks in Autonomous Driving8 Sep 2023
- 2023The TYC Dataset for Understanding Instance-Level Semantics and Motions of Cells in Microstructures23 Aug 2023
- 2023EndoDepthL: Lightweight Endoscopic Monocular Depth Estimation with CNN-Transformer4 Aug 2023
- 2023SplatFlow: Learning Multi-frame Optical Flow via Splatting15 Jun 2023
- 2023Unsupervised Learning Optical Flow in Multi-frame Dynamic Environment Using Temporal Dynamic Modeling14 Apr 2023
- 2023CapsFlow: Optical Flow Estimation with Capsule Networks1 Apr 2023
- 2023SemARFlow: Injecting Semantics into Unsupervised Optical Flow Estimation for Autonomous Driving10 Mar 2023
- 2023GyroFlow+: Gyroscope-Guided Unsupervised Deep Homography and Optical Flow Learning23 Jan 2023
- 2023Deep Planar Parallax for Monocular Depth Estimation9 Jan 2023
- 2022Self-supervised AutoFlow4 Dec 2022
- 2022MDFlow: Unsupervised Optical Flow Learning by Reliable Mutual Knowledge Distillation11 Nov 2022
- 2022Instance Segmentation with Cross-Modal Consistency14 Oct 2022
- 2022Rolling Shutter Inversion: Bring Rolling Shutter Images to High Framerate Global Shutter Video6 Oct 2022
- 2022Position-Aware Relation Learning for RGB-Thermal Salient Object Detection21 Sep 2022
- 2022INV-Flow2PoseNet: Light-Resistant Rigid Object Pose from Optical Flow of RGB-D Images using Images, Normals and Vertices14 Sep 2022
- 2022A Compacted Structure for Cross-domain learning on Monocular Depth and Flow Estimation25 Aug 2022
- 2022Unsupervised Flow Refinement near Motion Boundaries3 Aug 2022
- 2022Deep 360^circ Optical Flow Estimation Based on Multi-Projection Fusion27 Jul 2022
- 2022RealFlow: EM-based Realistic Optical Flow Dataset Generation from Videos22 Jul 2022
50 newest of 101 All 101 (JSON)
- 2021Learning to Estimate Hidden Motions with Global Motion Aggregationcited by 224
- 2022Simple Unsupervised Object-Centric Learning for Complex and Naturalistic Videoscited by 139
- 2024CoTracker3: Simpler and Better Point Tracking by Pseudo-Labelling Real Videoscited by 128
- 2021XVFI: eXtreme Video Frame Interpolationcited by 126
- 2024BootsTAP: Bootstrapped Training for Tracking-Any-Pointcited by 120
- 2021SMURF: Self-Teaching Multi-Frame Unsupervised RAFT with Full-Image Warpingcited by 74
- 2021Self-Supervised Monocular Depth and Ego-Motion Estimation in Endoscopy: Appearance Flow to the Rescuecited by 72
- 2021Learning Monocular Depth in Dynamic Scenes via Instance-Aware Projection Consistencycited by 58
- 2021Self-Supervised Pillar Motion Learning for Autonomous Drivingcited by 44
- 2021Warp Consistency for Unsupervised Learning of Dense Correspondencescited by 42
- 2022Bi-PointFlowNet: Bidirectional Learning for Point Cloud Based Scene Flow Estimationcited by 38
- 2022Learning Pixel Trajectories with Multiscale Contrastive Random Walkscited by 36
12 most cited of 97 All 97 (JSON)
- 2026Triangular Consistency as a Universal Constraint for Learning Optical Flow18 Jun 2026
- 2026Improved monocular depth prediction using distance transform over pre-semantic contours with self-supervised neural networks8 May 2026
- 2026Rebalancing gradient to improve self-supervised co-training of depth, odometry and optical flow predictions8 May 2026
- 2026RAFT-MSF++: Temporal Geometry-Motion Feature Fusion for Self-Supervised Monocular Scene Flow21 Apr 2026
- 2026U2Flow: Uncertainty-Aware Unsupervised Optical Flow Estimation11 Apr 2026
- 2026GenOpticalFlow: A Generative Approach to Unsupervised Optical Flow Learning23 Mar 2026
- 2025Visual Autoregressive Modelling for Monocular Depth Estimation27 Dec 2025
- 2025Dynamic Gaussian Splatting from Defocused and Motion-blurred Monocular Videos12 Oct 2025
- 2025EndoMUST: Monocular Depth Estimation for Robotic Endoscopy via End-to-end Multi-step Self-supervised Training19 Jun 2025
- 2025JointSplat: Probabilistic Joint Flow-Depth Optimization for Sparse-View Gaussian Splatting4 Jun 2025
- 2025AceVFI: A Comprehensive Survey of Advances in Video Frame Interpolation1 Jun 2025
- 2025Joint Flow And Feature Refinement Using Attention For Video Restoration22 May 2025
12 newest of 97 All 97 (JSON)