mlsanityData to 7 Oct 2026

Self-supervised Co-training for Video Representation Learning

arXiv 2010.09709 ↗First posted 19 Oct 2020cs.CVcited by 232

Not on the map. Why?

Abstract (arXiv)

The objective of this paper is visual-only self-supervised video representation learning. We make the following contributions: (i) we investigate the benefit of adding semantic-class positives to instance-based Info Noise Contrastive Estimation (InfoNCE) training, showing that this form of supervised contrastive learning leads to a clear improvement in performance; (ii) we propose a novel self-supervised co-training scheme to improve the popular infoNCE loss, exploiting the complementary information from different views, RGB streams and optical flow, of the same data source by using one view to obtain positive class samples for the other; (iii) we thoroughly evaluate the quality of the learnt representation on two different downstream tasks: action recognition and video retrieval. In both cases, the proposed approach demonstrates state-of-the-art or comparable performance with other self-supervised approaches, whilst being significantly more efficient to train, i.e. requiring far less training data to achieve similar performance.

Key facts. Cited by 232 papers in the map (data through 7 Oct 2026). First posted on arXiv 19 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 81; 2022: 60; 2023: 40; 2024: 27; 2025: 16; 2026: 8.

232 papers.

Sort:

By area:

  1. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,305
  2. 2023VideoMAE V2: Scaling Video Masked Autoencoders with Dual Maskingcited by 516
  3. 2021With a Little Help from My Friends: Nearest-Neighbor Contrastive Learning of Visual Representationscited by 324
  4. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 241
  5. 2021A Large-Scale Study on Unsupervised Spatiotemporal Representation Learningcited by 214
  6. 2023Learning to Exploit Temporal Structure for Biomedical Vision-Language Processingcited by 210
  7. 2023A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trendscited by 205
  8. 2021Self-supervised Video Object Segmentation by Motion Groupingcited by 147
  9. 2021Self-Supervised Representation Learning: Introduction, Advances and Challengescited by 147
  10. 2022Balanced Contrastive Learning for Long-Tailed Visual Recognitioncited by 143
  11. 2021TCLR: Temporal Contrastive Learning for Video Representationcited by 136
  12. 2021Broaden Your Views for Self-Supervised Video Learningcited by 108
38 more
  1. 20213D Human Action Representation Learning via Cross-View Consistency Pursuitcited by 105
  2. 2026V-JEPA 2.1: Unlocking Dense Features in Video Self-Supervised Learningcited by 104
  3. 2022In Defense of Online Models for Video Instance Segmentationcited by 97
  4. 2021CLOOB: Modern Hopfield Networks with InfoLOOB Outperform CLIPcited by 88
  5. 2022Self-Supervised Learning for Videos: A Surveycited by 85
  6. 2022Temporal Alignment Networks for Long-term Videocited by 84
  7. 2022CRAFT: Cross-Attentional Flow Transformer for Robust Optical Flowcited by 82
  8. 2021Robust Audio-Visual Instance Discriminationcited by 80
  9. 2022Object discovery and representation networkscited by 80
  10. 2022Bridging Video-text Retrieval with Multiple Choice Questionscited by 79
  11. 2021CrossCLR: Cross-modal Contrastive Learning For Multi-modal Video Representationscited by 72
  12. 2021Dual Contrastive Learning for Unsupervised Image-to-Image Translationcited by 65
  13. 2021Dissecting Supervised Contrastive Learningcited by 64
  14. 2021Self-supervised Video Transformercited by 64
  15. 2021Graph Contrastive Clusteringcited by 61
  16. 2023MomentDiff: Generative Video Moment Retrieval from Random to Realcited by 58
  17. 2021Home Action Genome: Cooperative Compositional Action Understandingcited by 52
  18. 2023Equivariant Similarity for Vision-Language Foundation Modelscited by 51
  19. 2021Improving Transferability of Representations via Augmentation-Aware Self-Supervisioncited by 48
  20. 2022Dissecting Self-Supervised Learning Methods for Surgical Computer Visioncited by 48
  21. 2021Learning Cross-modal Contrastive Features for Video Domain Adaptationcited by 47
  22. 2021Motion-aware Contrastive Video Representation Learning via Foreground-background Mergingcited by 45
  23. 2021Time-Equivariant Contrastive Video Representation Learningcited by 44
  24. 2022LocVTP: Video-Text Pre-training for Temporal Localizationcited by 44
  25. 2021Long-Short Temporal Contrastive Learning of Video Transformerscited by 42
  26. 2021Video Contrastive Learning with Global Contextcited by 40
  27. 2022Masked Motion Encoding for Self-Supervised Video Representation Learningcited by 39
  28. 2022Ranking Info Noise Contrastive Estimation: Boosting Contrastive Learning via Ranked Positivescited by 37
  29. 2021When Do Contrastive Learning Signals Help Spatio-Temporal Graph Forecasting?cited by 36
  30. 2021Shot Contrastive Self-Supervised Learning for Scene Boundary Detectioncited by 35
  31. 2021ASCNet: Self-supervised Video Representation Learning with Appearance-Speed Consistencycited by 35
  32. 2021Revitalizing CNN Attentions via Transformers in Self-Supervised Visual Representation Learningcited by 34
  33. 2022Probabilistic Representations for Video Contrastive Learningcited by 33
  34. 2021Multiview Pseudo-Labeling for Semi-supervised Learning from Videocited by 32
  35. 2022Learning Sound Localization Better From Semantically Similar Samplescited by 32
  36. 2023Learning Visual Representations via Language-Guided Samplingcited by 32
  37. 2021Space-Time Crop & Attend: Improving Cross-modal Video Representation Learningcited by 31
  38. 2022Unsupervised Pre-training for Temporal Action Localization Taskscited by 31

50 most cited of 232 All 232 (JSON)