# Back to the Future: Cycle Encoding Prediction for Self-supervised Contrastive Video Representation Learning

arXiv: https://arxiv.org/abs/2010.07217 · HTML: https://mlsanity.com/paper/2010.07217/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Oct 2020.

## Built on by (most cited first, 4 of 4)

- [Test of Time: Instilling Video-Language Models with a Sense of Time](https://mlsanity.com/paper/2301.02074/) (2023)
- [Self-supervised Spatiotemporal Representation Learning by Exploiting Video Continuity](https://mlsanity.com/paper/2112.05883/) (2021)
- [Unsupervised Visual Representation Learning by Tracking Patches in Video](https://mlsanity.com/paper/2105.02545/) (2021)
- [Auxiliary Learning for Self-Supervised Video Representation via Similarity-based Knowledge Distillation](https://mlsanity.com/paper/2112.04011/) (2021)

## Abstract (arXiv)

In this paper we show that learning video feature spaces in which temporal cycles are maximally predictable benefits action classification. In particular, we propose a novel learning approach termed Cycle Encoding Prediction (CEP) that is able to effectively represent high-level spatio-temporal structure of unlabelled video content. CEP builds a latent space wherein the concept of closed forward-backward as well as backward-forward temporal loops is approximately preserved. As a self-supervision signal, CEP leverages the bi-directional temporal coherence of the video stream and applies loss functions that encourage both temporal cycle closure as well as contrastive feature separation. Architecturally, the underpinning network structure utilises a single feature encoder for all video snippets, adding two predictive modules that learn temporal forward and backward transitions. We apply our framework for pretext training of networks for action recognition tasks. We report significantly improved results for the standard datasets UCF101 and HMDB51. Detailed ablation studies support the effectiveness of the proposed components. We publish source code for the CEP components in full with this paper.