Frequency Selective Augmentation for Video Representation Learning
Not on the map. Why?
Abstract (arXiv)
Recent self-supervised video representation learning methods focus on maximizing the similarity between multiple augmented views from the same video and largely rely on the quality of generated views. However, most existing methods lack a mechanism to prevent representation learning from bias towards static information in the video. In this paper, we propose frequency augmentation (FreqAug), a spatio-temporal data augmentation method in the frequency domain for video representation learning. FreqAug stochastically removes specific frequency components from the video so that learned representation captures essential features more from the remaining information for various downstream tasks. Specifically, FreqAug pushes the model to focus more on dynamic features rather than static features in the video via dropping spatial or temporal low-frequency components. To verify the generality of the proposed method, we experiment with FreqAug on multiple self-supervised learning frameworks along with standard augmentations. Transferring the improved representation to five video action recognition and two temporal action localization downstream tasks shows consistent improvements over baselines.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 55 of its 70 references that are in the map. First posted on arXiv 8 Apr 2022.
Builds on
55 of 70 references.
References · 55
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
- 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
- 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
- 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
- 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
- 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
- 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
- 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
- 2020Exploring Simple Siamese Representation Learningcited by 2,866
- 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
38 more
- 2020Improved Baselines with Momentum Contrastive Learningcited by 2,287
- 2018SlowFast Networks for Video Recognitioncited by 1,867
- 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,845
- 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,380
- 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,143
- 2017The "something something" video database for learning and evaluating visual common sensecited by 1,103
- 2016Temporal Segment Networks: Towards Good Practices for Deep Action Recognitioncited by 902
- 2020What Makes for Good Views for Contrastive Learning?cited by 846
- 2017Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?cited by 551
- 2017Large Batch Training of Convolutional Networkscited by 535
- 2017Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classificationcited by 521
- 2019End-to-End Learning of Visual Representations from Uncurated Instructional Videoscited by 507
- 2019MS-TCN: Multi-Stage Temporal Convolutional Network for Action Segmentationcited by 373
- 2020Spatiotemporal Contrastive Video Representation Learningcited by 372
- 2019High Frequency Component Helps Explain the Generalization of Convolutional Neural Networkscited by 310
- 2021A Fourier-based Framework for Domain Generalizationcited by 290
- 2020Self-Supervised MultiModal Versatile Networkscited by 276
- 2019Self-Supervised Learning by Cross-Modal Audio-Video Clusteringcited by 267
- 2019G-TAD: Sub-Graph Localization for Temporal Action Detectioncited by 249
- 2020FineGym: A Hierarchical Video Dataset for Fine-grained Action Understandingcited by 234
- 2020Self-supervised Co-training for Video Representation Learningcited by 232
- 2021A Large-Scale Study on Unsupervised Spatiotemporal Representation Learningcited by 214
- 2016The THUMOS Challenge on Action Recognition for Videos "in the Wild"cited by 213
- 2020How Well Do Self-Supervised Models Transfer?cited by 213
- 2021VideoMoCo: Contrastive Video Representation Learning with Temporally Adversarial Examplescited by 154
- 2020Robust and Generalizable Visual Representation Learning via Random Convolutionscited by 142
- 2021TCLR: Temporal Contrastive Learning for Video Representationcited by 136
- 2021Toward Understanding the Feature Learning Process of Self-supervised Contrastive Learningcited by 119
- 2020What makes instance discrimination good for transfer learning?cited by 111
- 2021Broaden Your Views for Self-Supervised Video Learningcited by 108
- 2020RSPNet: Relative Speed Perception for Unsupervised Video Representation Learningcited by 83
- 2018NSML: Meet the MLaaS platform with a real-world case studycited by 70
- 2020Removing the Background by Adding the Background: Towards Background Robust Self-supervised Video Representation Learningcited by 66
- 2021Motion-aware Contrastive Video Representation Learning via Foreground-background Mergingcited by 45
- 2020Boundary-sensitive Pre-training for Temporal Localization in Videoscited by 41
- 2020Enhancing Unsupervised Video Representation Learning by Decoupling the Scene and the Motioncited by 39
- 2021ASCNet: Self-supervised Video Representation Learning with Appearance-Speed Consistencycited by 35
- 2017NSML: A Machine Learning Platform That Enables You to Focus on Your Modelscited by 34
50 most cited of 55 All 55 (JSON)
Built on by
Citing papers per year: 2024: 2; 2025: 0; 2026: 1.
3 papers.