MoQuad: Motion-focused Quadruple Construction for Video Contrastive Learning
Not on the map. Why?
Abstract (arXiv)
Learning effective motion features is an essential pursuit of video representation learning. This paper presents a simple yet effective sample construction strategy to boost the learning of motion features in video contrastive learning. The proposed method, dubbed Motion-focused Quadruple Construction (MoQuad), augments the instance discrimination by meticulously disturbing the appearance and motion of both the positive and negative samples to create a quadruple for each video instance, such that the model is encouraged to exploit motion information. Unlike recent approaches that create extra auxiliary tasks for learning motion features or apply explicit temporal modelling, our method keeps the simple and clean contrastive learning paradigm (i.e.,SimCLR) without multi-task learning or extra modelling. In addition, we design two extra training strategies by analyzing initial MoQuad experiments. By simply applying MoQuad to SimCLR, extensive experiments show that we achieve superior performance on downstream tasks compared to the state of the arts. Notably, on the UCF-101 action recognition task, we achieve 93.7% accuracy after pre-training the model on Kinetics-400 for only 200 epochs, surpassing various previous methods
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 40 of its 48 references that are in the map. First posted on arXiv 21 Dec 2022.
Builds on
40 of 48 references.
References · 40
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
- 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
- 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
- 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
- 2020Exploring Simple Siamese Representation Learningcited by 2,866
- 2015Learning Deep Features for Discriminative Localizationcited by 2,436
- 2018SlowFast Networks for Video Recognitioncited by 1,867
- 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,591
- 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,382
- 2016Colorful Image Colorizationcited by 1,114
- 2017The "something something" video database for learning and evaluating visual common sensecited by 1,103
28 more
- 2017Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?cited by 551
- 2017Large Batch Training of Convolutional Networkscited by 535
- 2019Self-labelling via simultaneous clustering and representation learningcited by 528
- 2017Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classificationcited by 521
- 2018Bag of Tricks for Image Classification with Convolutional Neural Networkscited by 431
- 2020Spatiotemporal Contrastive Video Representation Learningcited by 372
- 2020Self-supervised Co-training for Video Representation Learningcited by 232
- 2021A Large-Scale Study on Unsupervised Spatiotemporal Representation Learningcited by 214
- 2019Video Representation Learning by Dense Predictive Codingcited by 175
- 2020SpeedNet: Learning the Speediness in Videoscited by 171
- 2021VideoMoCo: Contrastive Video Representation Learning with Temporally Adversarial Examplescited by 154
- 2020Self-supervised Video Representation Learning by Pace Predictioncited by 139
- 2020Memory-augmented Dense Predictive Coding for Video Representation Learningcited by 137
- 2020MAST: A Memory-Augmented Self-supervised Trackercited by 109
- 2015Learning to See by Movingcited by 102
- 2020Video Representation Learning by Recognizing Temporal Transformationscited by 83
- 2020RSPNet: Relative Speed Perception for Unsupervised Video Representation Learningcited by 83
- 2019Oops! Predicting Unintentional Action in Videocited by 82
- 2020Video Cloze Procedure for Self-Supervised Spatio-Temporal Learningcited by 81
- 2020Removing the Background by Adding the Background: Towards Background Robust Self-supervised Video Representation Learningcited by 66
- 2020Are all negatives created equal in contrastive instance discrimination?cited by 50
- 2019DynamoNet: Dynamic Action and Motion Networkcited by 48
- 2021Time-Equivariant Contrastive Video Representation Learningcited by 44
- 2015Learning image representations tied to ego-motioncited by 42
- 2015Learning visual groups from co-occurrences in space and timecited by 42
- 2021ASCNet: Self-supervised Video Representation Learning with Appearance-Speed Consistencycited by 35
- 2021Self-supervised Video Representation Learning by Context and Motion Decouplingcited by 34
- 2021Enhancing Self-supervised Video Representation Learning via Multi-level Feature Optimizationcited by 27
Built on by
Citing papers per year: 2023: 2.
2 papers.