mlsanityData to 14 Oct 2026

Contrastive Learning of Image Representations with Cross-Video Cycle-Consistency

arXiv 2105.06463 ↗First posted 13 May 2021cs.CVcited by 19

Not on the map. Why?

Abstract (arXiv)

Recent works have advanced the performance of self-supervised representation learning by a large margin. The core among these methods is intra-image invariance learning. Two different transformations of one image instance are considered as a positive sample pair, where various tasks are designed to learn invariant representations by comparing the pair. Analogically, for video data, representations of frames from the same video are trained to be closer than frames from other videos, i.e. intra-video invariance. However, cross-video relation has barely been explored for visual representation learning. Unlike intra-video invariance, ground-truth labels of cross-video relation is usually unavailable without human labors. In this paper, we propose a novel contrastive learning method which explores the cross-video relation by using cycle-consistency for general image representation learning. This allows to collect positive sample pairs across different video instances, which we hypothesize will lead to higher-level semantics. We validate our method by transferring our image representation to multiple downstream tasks including visual object tracking, image classification, and action recognition. We show significant improvement over state-of-the-art contrastive learning methods. Project page is available at https://happywu.github.io/cycle_contrast_video.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Builds on 66 of its 80 references that are in the map. First posted on arXiv 13 May 2021.

Builds on

66 of 80 references.

References · 66

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  4. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  5. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  6. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  7. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,287
  8. 2017The Kinetics Human Action Video Datasetcited by 1,909
  9. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,845
  10. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,591
  11. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,428
  12. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,382
38 more
  1. 2019Contrastive Multiview Codingcited by 1,301
  2. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,273
  3. 2016Colorful Image Colorizationcited by 1,114
  4. 2014Two-Stream Convolutional Networks for Action Recognition in Videoscited by 1,068
  5. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,066
  6. 2020What Makes for Good Views for Contrastive Learning?cited by 846
  7. 2016Unsupervised Monocular Depth Estimation with Left-Right Consistencycited by 820
  8. 2019Self-Supervised Learning of Pretext-Invariant Representationscited by 808
  9. 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 782
  10. 2019Learning Representations by Maximizing Mutual Information Across Viewscited by 734
  11. 2019Data-Efficient Image Recognition with Contrastive Predictive Codingcited by 673
  12. 2016Fully-Convolutional Siamese Networks for Object Trackingcited by 568
  13. 2018GOT-10k: A Large High-Diversity Benchmark for Generic Object Tracking in the Wildcited by 475
  14. 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 459
  15. 2020Spatiotemporal Contrastive Video Representation Learningcited by 372
  16. 2016Adversarial Feature Learningcited by 350
  17. 2018GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Posecited by 342
  18. 2014Discriminative Unsupervised Feature Learning with Exemplar Convolutional Neural Networkscited by 300
  19. 2019Learning Correspondence from the Cycle-Consistency of Timecited by 269
  20. 2019Self-Supervised Learning by Cross-Modal Audio-Video Clusteringcited by 267
  21. 2020Space-Time Correspondence as a Contrastive Random Walkcited by 239
  22. 2020What Should Not Be Contrastive in Contrastive Learningcited by 226
  23. 2017Unsupervised Representation Learning by Sorting Sequencescited by 221
  24. 2021A Large-Scale Study on Unsupervised Spatiotemporal Representation Learningcited by 214
  25. 2019Local Aggregation for Unsupervised Learning of Visual Embeddingscited by 204
  26. 2020Audio-Visual Instance Discrimination with Cross-Modal Agreementcited by 195
  27. 2019Video Representation Learning by Dense Predictive Codingcited by 175
  28. 2020SpeedNet: Learning the Speediness in Videoscited by 171
  29. 2019Temporal Cycle-Consistency Learningcited by 154
  30. 2018Self-Supervised Video Representation Learning with Space-Time Cubic Puzzlescited by 149
  31. 2020Demystifying Contrastive Self-Supervised Learning: Invariances, Augmentations and Dataset Biasescited by 143
  32. 2020Self-supervised Video Representation Learning by Pace Predictioncited by 139
  33. 2016Learning Features by Watching Objects Movecited by 138
  34. 2020Memory-augmented Dense Predictive Coding for Video Representation Learningcited by 137
  35. 2015Learning to See by Movingcited by 102
  36. 2016Learning Dense Correspondence via 3D-guided Cycle Consistencycited by 91
  37. 2018Recycle-GAN: Unsupervised Video Retargetingcited by 89
  38. 2020Video Representation Learning by Recognizing Temporal Transformationscited by 83

50 most cited of 66 All 66 (JSON)

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 5; 2023: 2; 2024: 7; 2025: 3.

19 papers.

Sort:

  1. 2021Targeted Supervised Contrastive Learning for Long-Tailed Recognitioncited by 132
  2. 2022Locality-Aware Inter-and Intra-Video Reconstruction for Self-Supervised Correspondence Learningcited by 26
  3. 2022Self-supervised video pretraining yields robust and more human-aligned visual representationscited by 20
  4. 2022Unsupervised Contrastive Learning of Image Representations from Ultrasound Videos with Hard Negative Miningcited by 14
  5. 2023ViC-MAE: Self-Supervised Representation Learning from Images and Video with Contrastive Masked Autoencoderscited by 14
  6. 2024Self-supervised visual learning from interactions with objectscited by 10
  7. 2024Enhanced Long-Tailed Recognition with Contrastive CutMix Augmentationcited by 7
  8. 2022Face-to-Face Contrastive Learning for Social Intelligence Question-Answeringcited by 5
  9. 2024Contrastive Learning of Person-independent Representations for Facial Action Unit Detectioncited by 5
  10. 2024S4: Self-Supervised Sensing Across the Spectrumcited by 5
  11. 2022Pixel-level Correspondence for Self-Supervised Learning from Videocited by 3
  12. 2025RefVNLI: Towards Scalable Evaluation of Subject-driven Text-to-image Generationcited by 3
7 more
  1. 2025Learning to See Through a Baby's Eyes: Early Visual Diets Enable Robust Visual Intelligence in Humans and Machinescited by 3
  2. 2023Boost Video Frame Interpolation via Motion Adaptationcited by 2
  3. 2024Learning Contrastive Feature Representations for Facial Action Unit Detectioncited by 2
  4. 2024Preventing Collapse in Contrastive Learning with Orthonormal Prototypes (CLOP)cited by 2
  5. 2021Mind Your Clever Neighbours: Unsupervised Person Re-identification via Adaptive Clustering Relationship Modelingcited by 0
  6. 2024Enhancing Hyperspectral Image Prediction with Contrastive Learning in Low-Label Regimecited by 0
  7. 2025Semi-Supervised Contrastive Learning with Orthonormal Prototypescited by 0