mlsanityData to 14 Oct 2026

Representation Learning with Contrastive Predictive Coding

arXiv 1807.03748 ↗First posted 10 Jul 2018cs.LG stat.MLcited by 7,777

Not on the map. Why?

Cited by 343 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

While supervised learning has enabled great progress in many applications, unsupervised learning has not seen such widespread adoption, and remains an important and challenging endeavor for artificial intelligence. In this work, we propose a universal unsupervised learning approach to extract useful representations from high-dimensional data, which we call Contrastive Predictive Coding. The key insight of our model is to learn such representations by predicting the future in latent space by using powerful autoregressive models. We use a probabilistic contrastive loss which induces the latent space to capture information that is maximally useful to predict future samples. It also makes the model tractable by using negative sampling. While most prior work has focused on evaluating representations for a particular modality, we demonstrate that our approach is able to learn useful representations achieving strong performance on four distinct domains: speech, images, text and reinforcement learning in 3D environments.

Key facts. Cited by 7,777 papers in the map (data through 14 Oct 2026). First posted on arXiv 10 Jul 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2019: 1; 2020: 0; 2021: 1015; 2022: 1325; 2023: 1415; 2024: 1374; 2025: 1328; 2026: 1319.

7,777 papers.

Sort:

By area:

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  3. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  4. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  5. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  6. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  7. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  8. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,838
  9. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
  10. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,619
  11. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,616
  12. 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,261
38 more
  1. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,252
  2. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,175
  3. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,173
  4. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,057
  5. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 1,039
  6. 2022Diffusion Models in Vision: A Surveycited by 1,015
  7. 2024Grounding Image Matching in 3D with MASt3Rcited by 983
  8. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 885
  9. 2023Vision-Language Models for Vision Tasks: A Surveycited by 839
  10. 2021MDETR -- Modulated Detection for End-to-End Multi-Modal Understandingcited by 819
  11. 2021Model-Contrastive Federated Learningcited by 766
  12. 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 727
  13. 2023Towards General Text Embeddings with Multi-stage Contrastive Learningcited by 723
  14. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 694
  15. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 666
  16. 2021Vector-quantized Image Modeling with Improved VQGANcited by 658
  17. 2021Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modelingcited by 653
  18. 2021RegionCLIP: Region-based Language-Image Pretrainingcited by 616
  19. 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 616
  20. 2021VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understandingcited by 602
  21. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 601
  22. 2021LiT: Zero-Shot Transfer with Locked-image text Tuningcited by 577
  23. 2022MedCLIP: Contrastive Learning from Unpaired Medical Images and Textcited by 570
  24. 2022Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learningcited by 561
  25. 2024Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inferencecited by 554
  26. 2021CutPaste: Self-Supervised Learning for Anomaly Detection and Localizationcited by 531
  27. 2024Revisiting Feature Prediction for Learning Visual Representations from Videocited by 531
  28. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learningcited by 523
  29. 2023BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairscited by 520
  30. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  31. 2023InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generationcited by 497
  32. 2023Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectivescited by 490
  33. 2023Exploiting Diffusion Prior for Real-World Image Super-Resolutioncited by 489
  34. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 478
  35. 2021Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesiscited by 464
  36. 2021VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretationcited by 463
  37. 2021Implicit Behavioral Cloningcited by 452
  38. 2021SLIP: Self-supervision meets Language-Image Pre-trainingcited by 436

50 most cited of 7,777 All 7,777 (JSON)