mlsanityData to 7 Oct 2026

Data-Efficient Image Recognition with Contrastive Predictive Coding

arXiv 1905.09272 ↗First posted 22 May 2019cs.CV cs.LGcited by 672

Not on the map. Why?

Abstract (arXiv)

Human observers can learn to recognize new categories of images from a handful of examples, yet doing so with artificial ones remains an open challenge. We hypothesize that data-efficient recognition is enabled by representations which make the variability in natural signals more predictable. We therefore revisit and improve Contrastive Predictive Coding, an unsupervised objective for learning such representations. This new implementation produces features which support state-of-the-art linear classification accuracy on the ImageNet dataset. When used as input for non-linear classification with deep neural networks, this representation allows us to use 2-5x less labels than classifiers trained directly on image pixels. Finally, this unsupervised representation substantially improves transfer learning to object detection on the PASCAL VOC dataset, surpassing fully supervised pre-trained ImageNet classifiers.

Key facts. Cited by 672 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 248; 2022: 174; 2023: 117; 2024: 70; 2025: 42; 2026: 21.

672 papers.

Sort:

By area:

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  3. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  4. 2021Learning to Prompt for Vision-Language Modelscited by 2,269
  5. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
  6. 2022Conditional Prompt Learning for Vision-Language Modelscited by 1,427
  7. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,147
  8. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,013
  9. 2021Transformers in Vision: A Surveycited by 1,009
  10. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 982
  11. 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 891
  12. 2021Vector-quantized Image Modeling with Improved VQGANcited by 656
38 more
  1. 2021Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesiscited by 464
  2. 2021VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Textcited by 427
  3. 2022Kubric: A scalable dataset generatorcited by 373
  4. 2021Bottleneck Transformers for Visual Recognitioncited by 364
  5. 2021With a Little Help from My Friends: Nearest-Neighbor Contrastive Learning of Visual Representationscited by 324
  6. 2021Big Self-Supervised Models Advance Medical Image Classificationcited by 320
  7. 2021Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Losscited by 311
  8. 2021Self-Supervised Learning with Data Augmentations Provably Isolates Content from Stylecited by 308
  9. 2022Scale-MAE: A Scale-Aware Masked Autoencoder for Multiscale Geospatial Representation Learningcited by 302
  10. 2022Context Autoencoder for Self-Supervised Representation Learningcited by 297
  11. 2023A Cookbook of Self-Supervised Learningcited by 248
  12. 2022Masked Visual Pre-training for Motor Controlcited by 246
  13. 2021Exploring Cross-Image Pixel Contrast for Semantic Segmentationcited by 242
  14. 2021Contrastive Learning Inverts the Data Generating Processcited by 220
  15. 2021Contrastive Learning for Compact Single Image Dehazingcited by 219
  16. 2021Self-Supervised Pretraining of 3D Features on any Point-Cloudcited by 217
  17. 2021A Large-Scale Study on Unsupervised Spatiotemporal Representation Learningcited by 214
  18. 2023A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trendscited by 205
  19. 2021Adversarial Graph Augmentation to Improve Graph Contrastive Learningcited by 200
  20. 2021Self-supervised Pretraining of Visual Features in the Wildcited by 197
  21. 2021BENDR: using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG datacited by 194
  22. 2022Unified Contrastive Learning in Image-Text-Label Spacecited by 194
  23. 2021Spatio-temporal Self-Supervised Representation Learning for 3D Point Cloudscited by 180
  24. 2021Reinforcement Learning with Prototypical Representationscited by 179
  25. 2021Unsupervised Degradation Representation Learning for Blind Super-Resolutioncited by 177
  26. 2021Behavior From the Void: Unsupervised Active Pre-Trainingcited by 173
  27. 2021A Good Image Generator Is What You Need for High-Resolution Video Synthesiscited by 162
  28. 2021On Feature Decorrelation in Self-Supervised Learningcited by 160
  29. 2022The Unsurprising Effectiveness of Pre-Trained Vision Models for Controlcited by 155
  30. 2021SCARF: Self-Supervised Contrastive Learning using Random Feature Corruptioncited by 151
  31. 2021Efficient Visual Pretraining with Contrastive Detectioncited by 150
  32. 2022Deep Spectral Methods: A Surprisingly Strong Baseline for Unsupervised Semantic Segmentation and Localizationcited by 150
  33. 2021Semantic Segmentation with Generative Models: Semi-Supervised Learning and Strong Out-of-Domain Generalizationcited by 141
  34. 2021URLB: Unsupervised Reinforcement Learning Benchmarkcited by 138
  35. 2022Self-supervised Learning in Remote Sensing: A Reviewcited by 137
  36. 2021SEED: Self-supervised Distillation For Visual Representationcited by 132
  37. 2022Leveraging Real Talking Faces via Self-Supervision for Robust Forgery Detectioncited by 130
  38. 2022Image-to-Lidar Self-Supervised Distillation for Autonomous Driving Datacited by 126

50 most cited of 672 All 672 (JSON)