# Improving Spatiotemporal Self-Supervision by Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/1807.11293 · HTML: https://mlsanity.com/paper/1807.11293/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Jul 2018.

## Built on by (most cited first, 30 of 35)

- [A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trends](https://mlsanity.com/paper/2301.05712/) (2023)
- [VideoMoCo: Contrastive Video Representation Learning with Temporally Adversarial Examples](https://mlsanity.com/paper/2103.05905/) (2021)
- [TCLR: Temporal Contrastive Learning for Video Representation](https://mlsanity.com/paper/2101.07974/) (2021)
- [Distilling Audio-Visual Knowledge by Compositional Contrastive Learning](https://mlsanity.com/paper/2104.10955/) (2021)
- [Representation Learning via Global Temporal Alignment and Cycle-Consistency](https://mlsanity.com/paper/2105.05217/) (2021)
- [Time-Equivariant Contrastive Video Representation Learning](https://mlsanity.com/paper/2112.03624/) (2021)
- [Video Contrastive Learning with Global Context](https://mlsanity.com/paper/2108.02722/) (2021)
- [Drop-DTW: Aligning Common Signal Between Sequences While Dropping Outliers](https://mlsanity.com/paper/2108.11996/) (2021)
- [ASCNet: Self-supervised Video Representation Learning with Appearance-Speed Consistency](https://mlsanity.com/paper/2106.02342/) (2021)
- [Self-supervised Video Representation Learning by Context and Motion Decoupling](https://mlsanity.com/paper/2104.00862/) (2021)
- [Action Shuffle Alternating Learning for Unsupervised Action Segmentation](https://mlsanity.com/paper/2104.02116/) (2021)
- [Space-Time Crop & Attend: Improving Cross-modal Video Representation Learning](https://mlsanity.com/paper/2103.10211/) (2021)
- [TCGL: Temporal Contrastive Graph for Self-supervised Video Representation Learning](https://mlsanity.com/paper/2112.03587/) (2021)
- [XKD: Cross-modal Knowledge Distillation with Domain Alignment for Video Representation Learning](https://mlsanity.com/paper/2211.13929/) (2022)
- [Self-Supervised Video Representation Learning with Meta-Contrastive Network](https://mlsanity.com/paper/2108.08426/) (2021)
- [Contrastive Learning of Image Representations with Cross-Video Cycle-Consistency](https://mlsanity.com/paper/2105.06463/) (2021)
- [Motion-Augmented Self-Training for Video Recognition at Smaller Scale](https://mlsanity.com/paper/2105.01646/) (2021)
- [SCVRL: Shuffled Contrastive Video Representation Learning](https://mlsanity.com/paper/2205.11710/) (2022)
- [STEPs: Self-Supervised Key Step Extraction and Localization from Unlabeled Procedural Videos](https://mlsanity.com/paper/2301.00794/) (2023)
- [Enhanced Multimodal Representation Learning with Cross-modal KD](https://mlsanity.com/paper/2306.07646/) (2023)
- [MaCLR: Motion-aware Contrastive Learning of Representations for Videos](https://mlsanity.com/paper/2106.09703/) (2021)
- [Progressive Scene Text Erasing with Self-Supervision](https://mlsanity.com/paper/2207.11469/) (2022)
- [Self-Supervised Audio-Visual Representation Learning with Relaxed Cross-Modal Synchronicity](https://mlsanity.com/paper/2111.05329/) (2021)
- [Temporal Contrastive Graph Learning for Video Action Recognition and Retrieval](https://mlsanity.com/paper/2101.00820/) (2021)
- [Self-supervised Video Representation Learning with Cross-Stream Prototypical Contrasting](https://mlsanity.com/paper/2106.10137/) (2021)
- [Self-Supervised Learning via multi-Transformation Classification for Action Recognition](https://mlsanity.com/paper/2102.10378/) (2021)
- [Improving Deep Metric Learning by Divide and Conquer](https://mlsanity.com/paper/2109.04003/) (2021)
- [Modeling Multiple Views via Implicitly Preserving Global Consistency and Local Complementarity](https://mlsanity.com/paper/2209.07811/) (2022)
- [PreViTS: Contrastive Pretraining with Video Tracking Supervision](https://mlsanity.com/paper/2112.00804/) (2021)
- [Information Theory-Guided Heuristic Progressive Multi-View Coding](https://mlsanity.com/paper/2109.02344/) (2021)

## Abstract (arXiv)

Self-supervised learning of convolutional neural networks can harness large amounts of cheap unlabeled data to train powerful feature representations. As surrogate task, we jointly address ordering of visual data in the spatial and temporal domain. The permutations of training samples, which are at the core of self-supervision by ordering, have so far been sampled randomly from a fixed preselected set. Based on deep reinforcement learning we propose a sampling policy that adapts to the state of the network, which is being trained. Therefore, new permutations are sampled according to their expected utility for updating the convolutional feature representation. Experimental evaluation on unsupervised and transfer learning tasks demonstrates competitive performance on standard benchmarks for image and video classification and nearest neighbor retrieval.