# Self-supervised Video Representation Learning Using Inter-intra Contrastive Framework

arXiv: https://arxiv.org/abs/2008.02531 · HTML: https://mlsanity.com/paper/2008.02531/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Aug 2020.

## Built on by (most cited first, 30 of 46)

- [TCLR: Temporal Contrastive Learning for Video Representation](https://mlsanity.com/paper/2101.07974/) (2021)
- [Self-Supervised Learning for Videos: A Survey](https://mlsanity.com/paper/2207.00419/) (2022)
- [iMiGUE: An Identity-free Video Dataset for Micro-Gesture Understanding and Emotion Analysis](https://mlsanity.com/paper/2107.00285/) (2021)
- [Long-Short Temporal Contrastive Learning of Video Transformers](https://mlsanity.com/paper/2106.09212/) (2021)
- [Video Contrastive Learning with Global Context](https://mlsanity.com/paper/2108.02722/) (2021)
- [TCGL: Temporal Contrastive Graph for Self-supervised Video Representation Learning](https://mlsanity.com/paper/2112.03587/) (2021)
- [Self-Supervised Video Representation Learning with Meta-Contrastive Network](https://mlsanity.com/paper/2108.08426/) (2021)
- [Beyond Just Vision: A Review on Self-Supervised Representation Learning on Multimodal and Temporal Data](https://mlsanity.com/paper/2206.02353/) (2022)
- [Motion-Focused Contrastive Learning of Video Representations](https://mlsanity.com/paper/2201.04029/) (2022)
- [How Severe is Benchmark-Sensitivity in Video Self-Supervised Learning?](https://mlsanity.com/paper/2203.14221/) (2022)
- [CONVIQT: Contrastive Video Quality Estimator](https://mlsanity.com/paper/2206.14713/) (2022)
- [SLIC: Self-Supervised Learning with Iterative Clustering for Human Action Videos](https://mlsanity.com/paper/2206.12534/) (2022)
- [TimeBalance: Temporally-Invariant and Temporally-Distinctive Video Representations for Semi-Supervised Action Recognition](https://mlsanity.com/paper/2303.16268/) (2023)
- [Dual Contrastive Learning for Spatio-temporal Representation](https://mlsanity.com/paper/2207.05340/) (2022)
- [SIGMA: Sinkhorn-Guided Masked Video Modeling](https://mlsanity.com/paper/2407.15447/) (2024)
- [Cross-Architecture Self-supervised Video Representation Learning](https://mlsanity.com/paper/2205.13313/) (2022)
- [HyRSM++: Hybrid Relation Guided Temporal Set Matching for Few-shot Action Recognition](https://mlsanity.com/paper/2301.03330/) (2023)
- [Self-Supervised Video Object Segmentation by Motion-Aware Mask Propagation](https://mlsanity.com/paper/2107.12569/) (2021)
- [Unsupervised Few-Shot Action Recognition via Action-Appearance Aligned Meta-Adaptation](https://mlsanity.com/paper/2109.15317/) (2021)
- [Contrastive Spatio-Temporal Pretext Learning for Self-supervised Video Representation](https://mlsanity.com/paper/2112.08913/) (2021)
- [Controllable Augmentations for Video Representation Learning](https://mlsanity.com/paper/2203.16632/) (2022)
- [Fine-Grained Spatiotemporal Motion Alignment for Contrastive Video Representation Learning](https://mlsanity.com/paper/2309.00297/) (2023)
- [Knowledge Guided Semi-Supervised Learning for Quality Assessment of User Generated Videos](https://mlsanity.com/paper/2312.15425/) (2023)
- [Temporal Contrastive Graph Learning for Video Action Recognition and Retrieval](https://mlsanity.com/paper/2101.00820/) (2021)
- [Inter-intra Variant Dual Representations forSelf-supervised Video Recognition](https://mlsanity.com/paper/2107.01194/) (2021)
- [A Large-Scale Analysis on Self-Supervised Video Representation Learning](https://mlsanity.com/paper/2306.06010/) (2023)
- [ALBAR: Adversarial Learning approach to mitigate Biases in Action Recognition](https://mlsanity.com/paper/2502.00156/) (2025)
- [Self-supervised Contrastive Learning for Audio-Visual Action Recognition](https://mlsanity.com/paper/2204.13386/) (2022)
- [ConKI: Contrastive Knowledge Injection for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2306.15796/) (2023)
- [PSM: Learning Probabilistic Embeddings for Multi-scale Zero-Shot Soundscape Mapping](https://mlsanity.com/paper/2408.07050/) (2024)

## Abstract (arXiv)

We propose a self-supervised method to learn feature representations from videos. A standard approach in traditional self-supervised methods uses positive-negative data pairs to train with contrastive learning strategy. In such a case, different modalities of the same video are treated as positives and video clips from a different video are treated as negatives. Because the spatio-temporal information is important for video representation, we extend the negative samples by introducing intra-negative samples, which are transformed from the same anchor video by breaking temporal relations in video clips. With the proposed Inter-Intra Contrastive (IIC) framework, we can train spatio-temporal convolutional networks to learn video representations. There are many flexible options in our IIC framework and we conduct experiments by using several different configurations. Evaluations are conducted on video retrieval and video recognition tasks using the learned video representation. Our proposed IIC outperforms current state-of-the-art results by a large margin, such as 16.7% and 9.5% points improvements in top-1 accuracy on UCF101 and HMDB51 datasets for video retrieval, respectively. For video recognition, improvements can also be obtained on these two benchmark datasets. Code is available at https://github.com/BestJuly/Inter-intra-video-contrastive-learning.