# Exploring Relations in Untrimmed Videos for Self-Supervised Learning

arXiv: https://arxiv.org/abs/2008.02711 · HTML: https://mlsanity.com/paper/2008.02711/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Aug 2020.

## Built on by (most cited first, 11 of 11)

- [UATVR: Uncertainty-Adaptive Text-Video Retrieval](https://mlsanity.com/paper/2301.06309/) (2023)
- [PIMNet: A Parallel, Iterative and Mimicking Network for Scene Text Recognition](https://mlsanity.com/paper/2109.04145/) (2021)
- [Dense Semantic Contrast for Self-Supervised Visual Representation Learning](https://mlsanity.com/paper/2109.07756/) (2021)
- [Mask is All You Need: Rethinking Mask R-CNN for Dense and Arbitrary-Shaped Scene Text Detection](https://mlsanity.com/paper/2109.03426/) (2021)
- [STEPs: Self-Supervised Key Step Extraction and Localization from Unlabeled Procedural Videos](https://mlsanity.com/paper/2301.00794/) (2023)
- [CtxMIM: Context-Enhanced Masked Image Modeling for Remote Sensing Image Understanding](https://mlsanity.com/paper/2310.00022/) (2023)
- [Self-supervised Video Representation Learning with Cross-Stream Prototypical Contrasting](https://mlsanity.com/paper/2106.10137/) (2021)
- [MMF: Multi-Task Multi-Structure Fusion for Hierarchical Image Classification](https://mlsanity.com/paper/2107.00808/) (2021)
- [UNITS: Unsupervised Intermediate Training Stage for Scene Text Detection](https://mlsanity.com/paper/2205.04683/) (2022)
- [Towards Weakly Supervised End-to-end Learning for Long-video Action Recognition](https://mlsanity.com/paper/2311.17118/) (2023)
- [Understanding the Performance Plateau in Text-to-Video Retrieval: A Comprehensive Empirical and Linguistic Analysis](https://mlsanity.com/paper/2605.00826/) (2026)

## Abstract (arXiv)

Existing video self-supervised learning methods mainly rely on trimmed videos for model training. However, trimmed datasets are manually annotated from untrimmed videos. In this sense, these methods are not really self-supervised. In this paper, we propose a novel self-supervised method, referred to as Exploring Relations in Untrimmed Videos (ERUV), which can be straightforwardly applied to untrimmed videos (real unlabeled) to learn spatio-temporal features. ERUV first generates single-shot videos by shot change detection. Then a designed sampling strategy is used to model relations for video clips. The strategy is saved as our self-supervision signals. Finally, the network learns representations by predicting the category of relations between the video clips. ERUV is able to compare the differences and similarities of videos, which is also an essential procedure for action and video related tasks. We validate our learned models with action recognition and video retrieval tasks with three kinds of 3D CNNs. Experimental results show that ERUV is able to learn richer representations and it outperforms state-of-the-art self-supervised methods with significant margins.