mlsanityData to 7 Oct 2026

Broaden Your Views for Self-Supervised Video Learning

arXiv 2103.16559 ↗First posted 30 Mar 2021cs.CV cs.LGcited by 108

Not on the map. Why?

Abstract (arXiv)

Most successful self-supervised learning methods are trained to align the representations of two independent views from the data. State-of-the-art methods in video are inspired by image techniques, where these two views are similarly extracted by cropping and augmenting the resulting crop. However, these methods miss a crucial element in the video domain: time. We introduce BraVe, a self-supervised learning framework for video. In BraVe, one of the views has access to a narrow temporal window of the video while the other view has a broad access to the video content. Our models learn to generalise from the narrow view to the general content of the video. Furthermore, BraVe processes the views with different backbones, enabling the use of alternative augmentations or modalities into the broad view such as optical flow, randomly convolved RGB frames, audio or their combinations. We demonstrate that BraVe achieves state-of-the-art results in self-supervised representation learning on standard video and audio classification benchmarks including UCF101, HMDB51, Kinetics, ESC-50 and AudioSet.

Key facts. Cited by 108 papers in the map (data through 7 Oct 2026). Builds on 76 of its 95 references that are in the map. First posted on arXiv 30 Mar 2021.

Builds on

76 of 95 references.

References · 76

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  5. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  7. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  8. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  9. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  10. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  11. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  12. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
38 more
  1. 2017The Kinetics Human Action Video Datasetcited by 1,894
  2. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,609
  3. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,586
  4. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,424
  5. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
  6. 2019Contrastive Multiview Codingcited by 1,297
  7. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,265
  8. 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,139
  9. 2016Colorful Image Colorizationcited by 1,113
  10. 2014Two-Stream Convolutional Networks for Action Recognition in Videoscited by 1,067
  11. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,062
  12. 2018Deep Clustering for Unsupervised Learning of Visual Featurescited by 1,020
  13. 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 946
  14. 2018TSM: Temporal Shift Module for Efficient Video Understandingcited by 852
  15. 2020What Makes for Good Views for Contrastive Learning?cited by 843
  16. 2019Self-Supervised Learning of Pretext-Invariant Representationscited by 806
  17. 2015Unsupervised Deep Embedding for Clustering Analysiscited by 763
  18. 2019Learning Representations by Maximizing Mutual Information Across Viewscited by 733
  19. 2019VideoBERT: A Joint Model for Video and Language Representation Learningcited by 702
  20. 2019Data-Efficient Image Recognition with Contrastive Predictive Codingcited by 672
  21. 2019PANNs: Large-Scale Pretrained Audio Neural Networks for Audio Pattern Recognitioncited by 563
  22. 2019Self-labelling via simultaneous clustering and representation learningcited by 525
  23. 2017Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classificationcited by 521
  24. 2019End-to-End Learning of Visual Representations from Uncurated Instructional Videoscited by 506
  25. 2017Look, Listen and Learncited by 452
  26. 2015Unsupervised Learning of Video Representations using LSTMscited by 434
  27. 2016Generating Videos with Scene Dynamicscited by 376
  28. 2020Spatiotemporal Contrastive Video Representation Learningcited by 372
  29. 2018Audio-Visual Scene Analysis with Self-Supervised Multisensory Featurescited by 335
  30. 2020Keystroke Dynamics as Part of Lifeloggingcited by 334
  31. 2018A Short Note about Kinetics-600cited by 331
  32. 2016YouTube-8M: A Large-Scale Video Classification Benchmarkcited by 324
  33. 2014Discriminative Unsupervised Feature Learning with Exemplar Convolutional Neural Networkscited by 299
  34. 2015Deep multi-scale video prediction beyond mean square errorcited by 297
  35. 2020Self-Supervised MultiModal Versatile Networkscited by 276
  36. 2019Self-Supervised Learning by Cross-Modal Audio-Video Clusteringcited by 264
  37. 2017Objects that Soundcited by 254
  38. 2018Cooperative Learning of Audio and Video Models from Self-Supervised Synchronizationcited by 246

50 most cited of 76 All 76 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 15; 2022: 37; 2023: 30; 2024: 14; 2025: 7; 2026: 5.

108 papers.

Sort:

By area:

  1. 2022Masked Autoencoders As Spatiotemporal Learnerscited by 428
  2. 2021VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Textcited by 427
  3. 2021Wav2CLIP: Learning Robust Audio Representations From CLIPcited by 210
  4. 2023Learning to Exploit Temporal Structure for Biomedical Vision-Language Processingcited by 210
  5. 2023A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trendscited by 205
  6. 2022Contrastive Audio-Visual Masked Autoencodercited by 142
  7. 2022Leveraging Real Talking Faces via Self-Supervision for Robust Forgery Detectioncited by 130
  8. 2023Selective Structured State-Spaces for Long-Form Video Understandingcited by 105
  9. 2024VideoPrism: A Foundational Visual Encoder for Video Understandingcited by 103
  10. 2021BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recognitioncited by 88
  11. 2022Object discovery and representation networkscited by 80
  12. 2023Verbs in Action: Improving verb understanding in video-language modelscited by 76
38 more
  1. 2023Siamese Masked Autoencoderscited by 75
  2. 2022BYOL-Explore: Exploration by Bootstrapped Predictioncited by 71
  3. 2023HierVL: Learning Hierarchical Video-Language Embeddingscited by 66
  4. 2022Audio Self-supervised Learning: A Surveycited by 65
  5. 2021Self-supervised Video Transformercited by 64
  6. 2022Video Transformers: A Surveycited by 63
  7. 2022MAViL: Masked Audio-Video Learnerscited by 56
  8. 2023MC-JEPA: A Joint-Embedding Predictive Architecture for Self-Supervised Learning of Motion and Content Featurescited by 53
  9. 2023Test of Time: Instilling Video-Language Models with a Sense of Timecited by 51
  10. 2021Contrast and Mix: Temporal Contrastive Video Domain Adaptation with Background Mixingcited by 49
  11. 2023Procedure-Aware Pretraining for Instructional Video Understandingcited by 48
  12. 2021Motion-aware Contrastive Video Representation Learning via Foreground-background Mergingcited by 45
  13. 2022Jointly Learning Visual and Auditory Speech Representations from Raw Datacited by 43
  14. 2021Long-Short Temporal Contrastive Learning of Video Transformerscited by 42
  15. 2022Masked Motion Encoding for Self-Supervised Video Representation Learningcited by 39
  16. 2022XKD: Cross-modal Knowledge Distillation with Domain Alignment for Video Representation Learningcited by 30
  17. 2023Motion-Guided Masking for Spatiotemporal Representation Learningcited by 30
  18. 2024Scaling 4D Representationscited by 30
  19. 2025AToken: A Unified Tokenizer for Visioncited by 30
  20. 2023VicTR: Video-conditioned Text Representations for Activity Recognitioncited by 28
  21. 2021Noether Networks: Meta-Learning Useful Conserved Quantitiescited by 26
  22. 2022How Severe is Benchmark-Sensitivity in Video Self-Supervised Learning?cited by 23
  23. 2022Static and Dynamic Concepts for Self-supervised Video Representation Learningcited by 23
  24. 2022Learning State-Aware Visual Representations from Audible Interactionscited by 23
  25. 2023Opening the Vocabulary of Egocentric Actionscited by 23
  26. 2022EgoEnv: Human-centric environment representations from egocentric videocited by 22
  27. 2021ViewCLR: Learning Self-supervised Video Representation for Unseen Viewpointscited by 20
  28. 2022Self-supervised video pretraining yields robust and more human-aligned visual representationscited by 20
  29. 2022Drive&Segment: Unsupervised Semantic Segmentation of Urban Scenes via Cross-modal Distillationcited by 19
  30. 2022Hierarchical Self-supervised Representation Learning for Movie Understandingcited by 19
  31. 2023Audio-Visual Contrastive Learning with Temporal Self-Supervisioncited by 18
  32. 2023COMEDIAN: Self-Supervised Learning and Knowledge Distillation for Action Spotting using Transformerscited by 17
  33. 2021Contextualized Spatio-Temporal Contrastive Learning with Self-Supervisioncited by 15
  34. 2022Dual Contrastive Learning for Spatio-temporal Representationcited by 15
  35. 2022TempCLR: Temporal Alignment Representation with Contrastive Learningcited by 15
  36. 2022SCVRL: Shuffled Contrastive Video Representation Learningcited by 14
  37. 2025CAV-MAE Sync: Improving Contrastive Audio-Visual Mask Autoencoders via Fine-Grained Alignmentcited by 14
  38. 2023STEPs: Self-Supervised Key Step Extraction and Localization from Unlabeled Procedural Videoscited by 13

50 most cited of 108 All 108 (JSON)