# Improvements to context based self-supervised learning

arXiv: https://arxiv.org/abs/1711.06379 · HTML: https://mlsanity.com/paper/1711.06379/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Nov 2017.

## Built on by (most cited first, 30 of 32)

- [A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trends](https://mlsanity.com/paper/2301.05712/) (2023)
- [Unsupervised Semantic Segmentation by Contrasting Object Mask Proposals](https://mlsanity.com/paper/2102.06191/) (2021)
- [Efficient Visual Pretraining with Contrastive Detection](https://mlsanity.com/paper/2103.10957/) (2021)
- [Self-supervised Learning in Remote Sensing: A Review](https://mlsanity.com/paper/2206.13188/) (2022)
- [Large-scale Unsupervised Semantic Segmentation](https://mlsanity.com/paper/2106.03149/) (2021)
- [Self-Supervised Learning for Videos: A Survey](https://mlsanity.com/paper/2207.00419/) (2022)
- [Object discovery and representation networks](https://mlsanity.com/paper/2203.08777/) (2022)
- [Transferable Visual Words: Exploiting the Semantics of Anatomical Patterns for Self-supervised Learning](https://mlsanity.com/paper/2102.10680/) (2021)
- [Equivariant Contrastive Learning](https://mlsanity.com/paper/2111.00899/) (2021)
- [Semi-Supervised and Unsupervised Deep Visual Learning: A Survey](https://mlsanity.com/paper/2208.11296/) (2022)
- [Unsupervised Discovery of the Long-Tail in Instance Segmentation Using Hierarchical Self-Supervision](https://mlsanity.com/paper/2104.01257/) (2021)
- [Jigsaw Clustering for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/2104.00323/) (2021)
- [Exploring Set Similarity for Dense Self-supervised Representation Learning](https://mlsanity.com/paper/2107.08712/) (2021)
- [Self-supervised Motion Learning from Static Images](https://mlsanity.com/paper/2104.00240/) (2021)
- [Location-Aware Self-Supervised Transformers for Semantic Segmentation](https://mlsanity.com/paper/2212.02400/) (2022)
- [DropPos: Pre-Training Vision Transformers by Reconstructing Dropped Positions](https://mlsanity.com/paper/2309.03576/) (2023)
- [Bayesian Imitation Learning for End-to-End Mobile Manipulation](https://mlsanity.com/paper/2202.07600/) (2022)
- [SegDiscover: Visual Concept Discovery via Unsupervised Semantic Segmentation](https://mlsanity.com/paper/2204.10926/) (2022)
- [ScatSimCLR: self-supervised contrastive learning with pretext task regularization for small-scale datasets](https://mlsanity.com/paper/2108.13939/) (2021)
- [Self-Supervised Features Improve Open-World Learning](https://mlsanity.com/paper/2102.07848/) (2021)
- [Global-Local Self-Distillation for Visual Representation Learning](https://mlsanity.com/paper/2207.14676/) (2022)
- [Practical Imitation Learning in the Real World via Task Consistency Loss](https://mlsanity.com/paper/2202.01862/) (2022)
- [Tailoring Self-Supervision for Supervised Learning](https://mlsanity.com/paper/2207.10023/) (2022)
- [Self-supervised Regularization for Text Classification](https://mlsanity.com/paper/2103.05231/) (2021)
- [Self-supervised Learning with Local Attention-Aware Feature](https://mlsanity.com/paper/2108.00475/) (2021)
- [Adaptive Similarity Bootstrapping for Self-Distillation based Representation Learning](https://mlsanity.com/paper/2303.13606/) (2023)
- [When Better Features Mean Greater Risks: The Performance-Privacy Trade-Off in Contrastive Learning](https://mlsanity.com/paper/2506.05743/) (2025)
- [How PARTs assemble into wholes: Learning the relative composition of images](https://mlsanity.com/paper/2506.03682/) (2025)
- [Fine-grained Hand Gesture Recognition in Multi-viewpoint Hand Hygiene](https://mlsanity.com/paper/2109.02917/) (2021)
- [Concurrent Discrimination and Alignment for Self-Supervised Feature Learning](https://mlsanity.com/paper/2108.08562/) (2021)

## Abstract (arXiv)

We develop a set of methods to improve on the results of self-supervised learning using context. We start with a baseline of patch based arrangement context learning and go from there. Our methods address some overt problems such as chromatic aberration as well as other potential problems such as spatial skew and mid-level feature neglect. We prevent problems with testing generalization on common self-supervised benchmark tests by using different datasets during our development. The results of our methods combined yield top scores on all standard self-supervised benchmarks, including classification and detection on PASCAL VOC 2007, segmentation on PASCAL VOC 2012, and "linear tests" on the ImageNet and CSAIL Places datasets. We obtain an improvement over our baseline method of between 4.0 to 7.1 percentage points on transfer learning classification tests. We also show results on different standard network architectures to demonstrate generalization as well as portability. All data, models and programs are available at: https://gdo-datasci.llnl.gov/selfsupervised/.