mlsanityData to 7 Oct 2026

CLIP-FLow: Contrastive Learning by semi-supervised Iterative Pseudo labeling for Optical Flow Estimation

arXiv 2210.14383 ↗First posted 25 Oct 2022cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

Synthetic datasets are often used to pretrain end-to-end optical flow networks, due to the lack of a large amount of labeled, real-scene data. But major drops in accuracy occur when moving from synthetic to real scenes. How do we better transfer the knowledge learned from synthetic to real domains? To this end, we propose CLIP-FLow, a semi-supervised iterative pseudo-labeling framework to transfer the pretraining knowledge to the target real domain. We leverage large-scale, unlabeled real data to facilitate transfer learning with the supervision of iteratively updated pseudo-ground truth labels, bridging the domain gap between the synthetic and the real. In addition, we propose a contrastive flow loss on reference features and the warped features by pseudo ground truth flows, to further boost the accurate matching and dampen the mismatching due to motion, occlusion, or noisy pseudo labels. We adopt RAFT as the backbone and obtain an F1-all error of 4.11%, i.e. a 19% error reduction from RAFT (5.10%) and ranking 2nd place at submission on the KITTI 2015 benchmark. Our framework can also be extended to other models, e.g. CRAFT, reducing the F1-all error from 4.79% to 4.66% on KITTI 2015 benchmark.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 43 of its 60 references that are in the map. First posted on arXiv 25 Oct 2022.

Builds on

43 of 60 references.

References · 43

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  3. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  4. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  5. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  6. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  7. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  8. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  9. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  10. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  11. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
  12. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
31 more
  1. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,265
  2. 2014Two-Stream Convolutional Networks for Action Recognition in Videoscited by 1,067
  3. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,054
  4. 2017PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volumecited by 966
  5. 2015A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimationcited by 878
  6. 2016FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networkscited by 780
  7. 2019Learning Representations by Maximizing Mutual Information Across Viewscited by 733
  8. 2016Optical Flow Estimation using a Spatial Pyramid Networkcited by 421
  9. 2018An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solutioncited by 302
  10. 2022FlowFormer: A Transformer Architecture for Optical Flowcited by 273
  11. 2021Learning to Estimate Hidden Motions with Global Motion Aggregationcited by 223
  12. 2020ClassMix: Segmentation-Based Data Augmentation for Semi-Supervised Learningcited by 209
  13. 2021DetCo: Unsupervised Contrastive Learning for Object Detectioncited by 205
  14. 2017Computer Vision for Autonomous Vehicles: Problems, Datasets and State of the Artcited by 182
  15. 2018DF-Net: Unsupervised Joint Learning of Depth and Flow using Cross-Task Consistencycited by 148
  16. 2020Unsupervised Learning of Dense Visual Representationscited by 147
  17. 2021Self-supervised Video Object Segmentation by Motion Groupingcited by 147
  18. 2021Self-Supervised Learning with Swin Transformerscited by 121
  19. 2020MaskFlownet: Asymmetric Feature Matching with Learnable Occlusion Maskcited by 117
  20. 2021AutoFlow: Learning a Better Training Set for Optical Flowcited by 102
  21. 2021Region Similarity Representation Learningcited by 96
  22. 2018Models Matter, So Does Training: An Empirical Study of CNNs for Optical Flow Estimationcited by 94
  23. 2017Optical Flow Guided Feature: A Fast and Robust Motion Representation for Video Action Recognitioncited by 82
  24. 2022CRAFT: Cross-Attentional Flow Transformer for Robust Optical Flowcited by 82
  25. 2019A Lightweight Optical Flow CNN - Revisiting Data Fidelity and Regularizationcited by 76
  26. 2020Three Ways to Improve Semantic Segmentation with Self-Supervised Depth Estimationcited by 66
  27. 2022Patch-level Representation Learning for Self-supervised Vision Transformerscited by 60
  28. 2021Learning to Segment Rigid Motions from Two Framescited by 45
  29. 2017On the Integration of Optical Flow and Action Recognitioncited by 42
  30. 2020Matching-space Stereo Networks for Cross-domain Generalizationcited by 32
  31. 2022Imposing Consistency for Optical Flow Estimationcited by 24

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 1; 2025: 0; 2026: 1.

4 papers.

  1. 2023ADFactory: An Effective Framework for Generalizing Optical Flow with Nerfcited by 9
  2. 2023UP-DP: Unsupervised Prompt Learning for Data Pre-Selection with Vision-Language Modelscited by 3
  3. 2024Self-Assessed Generation: Trustworthy Label Generation for Optical Flow and Stereo Matching in Real-worldcited by 3
  4. 2026GenOpticalFlow: A Generative Approach to Unsupervised Optical Flow Learningcited by 0