mlsanityData to 14 Oct 2026

Unsupervised Visual Representation Learning by Context Prediction

arXiv 1505.05192 ↗First posted 19 May 2015cs.CVcited by 1,066

Not on the map. Why?

Cited by 20 in Knowledge distillation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

This work explores the use of spatial context as a source of free and plentiful supervisory signal for training a rich visual representation. Given only a large, unlabeled image collection, we extract random pairs of patches from each image and train a convolutional neural net to predict the position of the second patch relative to the first. We argue that doing well on this task requires the model to learn to recognize objects and their parts. We demonstrate that the feature representation learned using this within-image context indeed captures visual similarity across images. For example, this representation allows us to perform unsupervised visual discovery of objects like cats, people, and even birds from the Pascal VOC 2011 detection dataset. Furthermore, we show that the learned ConvNet can be used in the R-CNN framework and provides a significant boost over a randomly-initialized ConvNet, resulting in state-of-the-art performance among algorithms which use only Pascal-provided training set annotations.

Key facts. Cited by 1,066 papers in the map (data through 14 Oct 2026). First posted on arXiv 19 May 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2020: 2; 2021: 345; 2022: 290; 2023: 216; 2024: 112; 2025: 59; 2026: 42.

1,066 papers.

Sort:

By area:

  1. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  2. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  3. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,153
  4. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,057
  5. 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 903
  6. 2023Vision Transformers Need Registerscited by 871
  7. 2021Robust fine-tuning of zero-shot modelscited by 797
  8. 2022AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognitioncited by 719
  9. 2021Vector-quantized Image Modeling with Improved VQGANcited by 658
  10. 2021FILIP: Fine-grained Interactive Language-Image Pre-Trainingcited by 627
  11. 2021Extract Free Dense Labels from CLIPcited by 583
  12. 2021LiT: Zero-Shot Transfer with Locked-image text Tuningcited by 577
38 more
  1. 2021Masked Feature Prediction for Self-Supervised Visual Pre-Trainingcited by 576
  2. 2022InternVideo: General Video Foundation Models via Generative and Discriminative Learningcited by 447
  3. 2021VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Textcited by 429
  4. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 372
  5. 2023A Comprehensive Survey on Test-Time Adaptation under Distribution Shiftscited by 359
  6. 2021With a Little Help from My Friends: Nearest-Neighbor Contrastive Learning of Visual Representationscited by 326
  7. 2021Big Self-Supervised Models Advance Medical Image Classificationcited by 320
  8. 2021Self-Supervised Learning with Data Augmentations Provably Isolates Content from Stylecited by 312
  9. 2021In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learningcited by 301
  10. 2021Layer-wise Analysis of a Self-supervised Speech Representation Modelcited by 298
  11. 2022Context Autoencoder for Self-Supervised Representation Learningcited by 297
  12. 2022Self-Supervised Speech Representation Learning: A Reviewcited by 260
  13. 2023A Cookbook of Self-Supervised Learningcited by 250
  14. 2022Masked Visual Pre-training for Motor Controlcited by 248
  15. 2022Real-World Robot Learning with Masked Visual Pre-trainingcited by 245
  16. 2021Exploring Cross-Image Pixel Contrast for Semantic Segmentationcited by 242
  17. 2022Unsupervised Semantic Segmentation by Distilling Feature Correspondencescited by 241
  18. 2021Self-Supervised Pretraining of 3D Features on any Point-Cloudcited by 217
  19. 2021Seasonal Contrast: Unsupervised Pre-Training from Uncurated Remote Sensing Datacited by 215
  20. 2021A Large-Scale Study on Unsupervised Spatiotemporal Representation Learningcited by 214
  21. 2022Vision-Language Pre-Training with Triple Contrastive Learningcited by 212
  22. 2023Sequential Modeling Enables Scalable Learning for Large Vision Modelscited by 212
  23. 2023A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trendscited by 207
  24. 2021DetCo: Unsupervised Contrastive Learning for Object Detectioncited by 205
  25. 2021ChemRL-GEM: Geometry Enhanced Molecular Representation Learning for Property Predictioncited by 204
  26. 2021Self-supervised Pretraining of Visual Features in the Wildcited by 198
  27. 2021Unsupervised Semantic Segmentation by Contrasting Object Mask Proposalscited by 190
  28. 2023Cut and Learn for Unsupervised Object Detection and Instance Segmentationcited by 187
  29. 2021Self-Supervised Learning of Graph Neural Networks: A Unified Reviewcited by 184
  30. 2022MERLOT Reserve: Neural Script Knowledge through Vision and Language and Soundcited by 184
  31. 2023CLIP2Scene: Towards Label-efficient 3D Scene Understanding by CLIPcited by 181
  32. 2021Spatio-temporal Self-Supervised Representation Learning for 3D Point Cloudscited by 180
  33. 2021Reinforcement Learning with Prototypical Representationscited by 179
  34. 2021Unsupervised Degradation Representation Learning for Blind Super-Resolutioncited by 177
  35. 2021Benchmarking Representation Learning for Natural World Image Collectionscited by 168
  36. 2021Open-World Semi-Supervised Learningcited by 165
  37. 2021PiCIE: Unsupervised Semantic Segmentation using Invariance and Equivariance in Clusteringcited by 163
  38. 2021SCARF: Self-Supervised Contrastive Learning using Random Feature Corruptioncited by 156

50 most cited of 1,066 All 1,066 (JSON)