# Unsupervised Video Representation Learning by Bidirectional Feature Prediction

arXiv: https://arxiv.org/abs/2011.06037 · HTML: https://mlsanity.com/paper/2011.06037/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Nov 2020.

## Built on by (most cited first, 17 of 17)

- [TCLR: Temporal Contrastive Learning for Video Representation](https://mlsanity.com/paper/2101.07974/) (2021)
- [Iso-Dream: Isolating and Leveraging Noncontrollable Visual Dynamics in World Models](https://mlsanity.com/paper/2205.13817/) (2022)
- [Self-Supervised Material and Texture Representation Learning for Remote Sensing Tasks](https://mlsanity.com/paper/2112.01715/) (2021)
- [Video Contrastive Learning with Global Context](https://mlsanity.com/paper/2108.02722/) (2021)
- [Ranking Info Noise Contrastive Estimation: Boosting Contrastive Learning via Ranked Positives](https://mlsanity.com/paper/2201.11736/) (2022)
- [Enhancing Self-supervised Video Representation Learning via Multi-level Feature Optimization](https://mlsanity.com/paper/2108.02183/) (2021)
- [Static and Dynamic Concepts for Self-supervised Video Representation Learning](https://mlsanity.com/paper/2207.12795/) (2022)
- [Long Short View Feature Decomposition via Contrastive Video Representation Learning](https://mlsanity.com/paper/2109.11593/) (2021)
- [Rethinking InfoNCE: How Many Negative Samples Do You Need?](https://mlsanity.com/paper/2105.13003/) (2021)
- [Dual Contrastive Learning for Spatio-temporal Representation](https://mlsanity.com/paper/2207.05340/) (2022)
- [Contrastive Spatio-Temporal Pretext Learning for Self-supervised Video Representation](https://mlsanity.com/paper/2112.08913/) (2021)
- [Controllable Augmentations for Video Representation Learning](https://mlsanity.com/paper/2203.16632/) (2022)
- [Model-Based Reinforcement Learning with Isolated Imaginations](https://mlsanity.com/paper/2303.14889/) (2023)
- [Human-Centered Prior-Guided and Task-Dependent Multi-Task Representation Learning for Action Recognition Pre-Training](https://mlsanity.com/paper/2204.12729/) (2022)
- [Self-Supervised Video Representation Learning by Video Incoherence Detection](https://mlsanity.com/paper/2109.12493/) (2021)
- [Spatio-Temporal Video Representation Learning for AI Based Video Playback Style Prediction](https://mlsanity.com/paper/2110.01015/) (2021)
- [Self-supervised and Weakly Supervised Contrastive Learning for Frame-wise Action Representations](https://mlsanity.com/paper/2212.03125/) (2022)

## Abstract (arXiv)

This paper introduces a novel method for self-supervised video representation learning via feature prediction. In contrast to the previous methods that focus on future feature prediction, we argue that a supervisory signal arising from unobserved past frames is complementary to one that originates from the future frames. The rationale behind our method is to encourage the network to explore the temporal structure of videos by distinguishing between future and past given present observations. We train our model in a contrastive learning framework, where joint encoding of future and past provides us with a comprehensive set of temporal hard negatives via swapping. We empirically show that utilizing both signals enriches the learned representations for the downstream task of action recognition. It outperforms independent prediction of future and past.