mlsanityData to 7 Oct 2026

Contrastive estimation reveals topic posterior information to linear models

arXiv 2003.02234 ↗First posted 4 Mar 2020cs.LG stat.MLcited by 54

Not on the map. Why?

Abstract (arXiv)

Contrastive learning is an approach to representation learning that utilizes naturally occurring similar and dissimilar pairs of data points to find useful embeddings of data. In the context of document classification under topic modeling assumptions, we prove that contrastive learning is capable of recovering a representation of documents that reveals their underlying topic posterior information to linear models. We apply this procedure in a semi-supervised setup and demonstrate empirically that linear classifiers with these representations perform well in document classification tasks with very few training examples.

Key facts. Cited by 54 papers in the map (data through 7 Oct 2026). First posted on arXiv 4 Mar 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 8; 2022: 15; 2023: 20; 2024: 7; 2025: 3; 2026: 1.

54 papers.

Sort:

By area:

  1. 2021Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Losscited by 311
  2. 2021Self-Supervised Learning with Data Augmentations Provably Isolates Content from Stylecited by 308
  3. 2021Self-supervised Learning is More Robust to Dataset Imbalancecited by 122
  4. 2021Toward Understanding the Feature Learning Process of Self-supervised Contrastive Learningcited by 119
  5. 2022A Kernel-Based View of Language Model Fine-Tuningcited by 118
  6. 2022Chaos is a Ladder: A New Theoretical Understanding of Contrastive Learning via Augmentation Overlapcited by 105
  7. 2022Understanding Contrastive Learning Requires Incorporating Inductive Biasescited by 96
  8. 2023How Do Transformers Learn Topic Structure: Towards a Mechanistic Understandingcited by 85
  9. 2021Why Do Pretrained Language Models Help in Downstream Tasks? An Analysis of Head and Prompt Tuningcited by 82
  10. 2022Connect, Not Collapse: Explaining Contrastive Learning for Unsupervised Domain Adaptationcited by 66
  11. 2021Investigating the Role of Negatives in Contrastive Representation Learningcited by 54
  12. 2022Investigating Why Contrastive Learning Benefits Robustness Against Label Noisecited by 46
38 more
  1. 2022Beyond Separability: Analyzing the Linear Transferability of Contrastive Representations to Related Subpopulationscited by 44
  2. 2023The SSL Interplay: Augmentations, Inductive Bias, and Generalizationcited by 40
  3. 2022The Mechanism of Prediction Head in Non-contrastive Self-supervised Learningcited by 38
  4. 2022A Theoretical Study of Inductive Biases in Contrastive Learningcited by 38
  5. 2023On the Stepwise Nature of Self-Supervised Learningcited by 36
  6. 2022Improving Self-Supervised Learning by Characterizing Idealized Representationscited by 33
  7. 2023Which Features are Learnt by Contrastive Learning? On the Role of Simplicity Bias in Class Collapse and Feature Suppressioncited by 30
  8. 2023Understanding Transferable Representation Learning and Zero-shot Transfer in CLIPcited by 26
  9. 2023Generalization Analysis for Contrastive Representation Learningcited by 25
  10. 2023Data-Efficient Contrastive Self-supervised Learning: Most Beneficial Examples for Supervised Learning Contribute the Leastcited by 22
  11. 2023A Graph-Theoretic Framework for Understanding Open-World Semi-Supervised Learningcited by 22
  12. 2023On the Provable Advantage of Unsupervised Pretrainingcited by 21
  13. 2023Matrix Information Theory for Self-Supervised Learningcited by 21
  14. 2023When and How Does Known Class Help Discover Unknown Ones? Provable Understanding Through Spectral Analysiscited by 21
  15. 2023Understanding Augmentation-based Self-Supervised Representation Learning via RKHS Approximation and Regressioncited by 20
  16. 2022What shapes the loss landscape of self-supervised learning?cited by 19
  17. 2022Towards Understanding Why Mask-Reconstruction Pretraining Helps in Downstream Taskscited by 18
  18. 2023Information Flow in Self-Supervised Learningcited by 16
  19. 2024Identifiable Causal Representation Learning: Unsupervised, Multi-View, and Multi-Environmentcited by 12
  20. 2021Leveraging Time Irreversibility with Order-Contrastive Pre-trainingcited by 7
  21. 2024Collapse-Proof Non-Contrastive Self-Supervised Learningcited by 7
  22. 2022GEDI: GEnerative and DIscriminative Training for Self-Supervised Learningcited by 6
  23. 2023Towards Understanding the Mechanism of Contrastive Learning via Similarity Structure: A Theoretical Analysiscited by 6
  24. 2025Provable Benefits of Unsupervised Pre-training and Transfer Learning via Single-Index Modelscited by 6
  25. 2022Empirical Evaluation and Theoretical Analysis for Representation Learning: A Surveycited by 5
  26. 2025An Augmentation Overlap Theory of Contrastive Learningcited by 5
  27. 2022Masked prediction tasks: a parameter identifiability viewcited by 4
  28. 2023Understanding the Robustness of Multi-modal Contrastive Learning to Distribution Shiftcited by 4
  29. 2025A Statistical Theory of Contrastive Learning via Approximate Sufficient Statisticscited by 4
  30. 2021Contrastive learning of strong-mixing continuous-time stochastic processescited by 3
  31. 2022Understanding The Robustness of Self-supervised Learning Through Topic Modelingcited by 3
  32. 2024Bridging OOD Detection and Generalization: A Graph-Theoretic Viewcited by 3
  33. 2023Spectral Neural Networks: Approximation Theory and Optimization Landscapecited by 2
  34. 2023Unifying Self-Supervised Clustering and Energy-Based Modelscited by 2
  35. 2024Understanding Representation Learnability of Nonlinear Self-Supervised Learningcited by 2
  36. 2024Beyond Spectral Decomposition: Bayesian Contrastive Learning and its Non-negative Formulation via Factor Analysiscited by 2
  37. 2023Graph Contrastive Topic Modelcited by 1
  38. 2023Detecting and Learning Out-of-Distribution Data in the Open world: Algorithm and Theorycited by 1

50 most cited of 54 All 54 (JSON)