mlsanityData to 7 Oct 2026

Your Contrastive Learning Is Secretly Doing Stochastic Neighbor Embedding

arXiv 2205.14814 ↗First posted 30 May 2022cs.LG cs.AI cs.CVcited by 28

Not on the map. Why?

Abstract (arXiv)

Contrastive learning, especially self-supervised contrastive learning (SSCL), has achieved great success in extracting powerful features from unlabeled data. In this work, we contribute to the theoretical understanding of SSCL and uncover its connection to the classic data visualization method, stochastic neighbor embedding (SNE), whose goal is to preserve pairwise distances. From the perspective of preserving neighboring information, SSCL can be viewed as a special case of SNE with the input space pairwise similarities specified by data augmentation. The established correspondence facilitates deeper theoretical understanding of learned features of SSCL, as well as methodological guidelines for practical improvement. Specifically, through the lens of SNE, we provide novel analysis on domain-agnostic augmentations, implicit bias and robustness of learned features. To illustrate the practical advantage, we demonstrate that the modifications from SNE to t-SNE can also be adopted in the SSCL setting, achieving significant improvement in both in-distribution and out-of-distribution generalization.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Builds on 60 of its 82 references that are in the map. First posted on arXiv 30 May 2022.

Builds on

60 of 82 references.

References · 60

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  6. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  7. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  8. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  9. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  10. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  11. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  12. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
38 more
  1. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
  2. 2019Invariant Risk Minimizationcited by 1,832
  3. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
  4. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,393
  5. 2019Contrastive Multiview Codingcited by 1,297
  6. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,265
  7. 2017Deep Hashing Network for Unsupervised Domain Adaptationcited by 1,181
  8. 2020In Search of Lost Domain Generalizationcited by 1,042
  9. 2017Deeper, Broader and Artier Domain Generalizationcited by 983
  10. 2020What Makes for Good Views for Contrastive Learning?cited by 843
  11. 2020Out-of-Distribution Generalization via Risk Extrapolation (REx)cited by 737
  12. 2019Learning Representations by Maximizing Mutual Information Across Viewscited by 733
  13. 2021FILIP: Fine-grained Interactive Language-Image Pre-Trainingcited by 626
  14. 2019A Theoretical Analysis of Contrastive Unsupervised Representation Learningcited by 530
  15. 2021Understanding Dimensional Collapse in Contrastive Self-supervised Learningcited by 374
  16. 2021Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Losscited by 311
  17. 2020Contrastive Clusteringcited by 264
  18. 2020DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representationscited by 249
  19. 2021ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfercited by 231
  20. 2021Contrastive Learning Inverts the Data Generating Processcited by 220
  21. 2020How Well Do Self-Supervised Models Transfer?cited by 213
  22. 2020CLEAR: Contrastive Learning for Sentence Representationcited by 186
  23. 2020Theoretical Analysis of Self-Training with Deep Networks on Unlabeled Datacited by 179
  24. 2020CERT: Contrastive Self-supervised Learning for Language Understandingcited by 169
  25. 2020Contrastive learning, multi-view redundancy, and linear modelscited by 146
  26. 2022Contrastive and Non-Contrastive Self-Supervised Learning Recover Global and Local Spectral Embedding Methodscited by 146
  27. 2021Decoupled Contrastive Learningcited by 141
  28. 2021Toward Understanding the Feature Learning Process of Self-supervised Contrastive Learningcited by 119
  29. 2020Implicit Regularization in Deep Learning May Not Be Explainable by Normscited by 118
  30. 2021Instance Localization for Self-supervised Detection Pretrainingcited by 99
  31. 2021Towards the Generalization of Contrastive Self-Supervised Learningcited by 99
  32. 2022Understanding Contrastive Learning Requires Incorporating Inductive Biasescited by 96
  33. 2020Towards Domain-Agnostic Contrastive Learningcited by 75
  34. 2020Implicit Bias in Deep Linear Classification: Initialization Scale vs Training Accuracycited by 70
  35. 2021On the Implicit Bias of Initialization Shape: Beyond Infinitesimal Mirror Descentcited by 69
  36. 2021Theoretical Foundations of t-SNE for Visualizing High-Dimensional Clustered Datacited by 55
  37. 2019Learning Generative Models across Incomparable Spacescited by 53
  38. 2021The Power of Contrast for Feature Learning: A Theoretical Analysiscited by 53

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 9; 2024: 5; 2025: 6; 2026: 5.

28 papers.

Sort:

By area:

  1. 2023Know Your Self-supervised Learning: A Survey on Image-based Generative and Discriminative Trainingcited by 39
  2. 2022From t-SNE to UMAP with contrastive learningcited by 28
  3. 2023Contrastive Learning Is Spectral Clustering On Similarity Graphcited by 24
  4. 2023Matrix Information Theory for Self-Supervised Learningcited by 21
  5. 2023Complexity Matters: Rethinking the Latent Space for Generative Modelingcited by 19
  6. 2023Rethinking Weak Supervision in Helping Contrastive Learningcited by 18
  7. 2022Unsupervised visualization of image datasets using contrastive learningcited by 16
  8. 2023Information Flow in Self-Supervised Learningcited by 16
  9. 2025I-Con: A Unifying Framework for Representation Learningcited by 15
  10. 2023Explore and Exploit the Diverse Knowledge in Model Zoo for Domain Generalizationcited by 14
  11. 2023ArCL: Enhancing Contrastive Learning with Augmentation-Robust Representationscited by 10
  12. 2022Augmentation Component Analysis: Modeling Similarity via the Augmentation Overlapscited by 5
16 more
  1. 2023Deciphering the Projection Head: Representation Evaluation Self-supervised Learningcited by 5
  2. 2025The Shape of Attraction in UMAP: Exploring the Embedding Forces in Dimensionality Reductioncited by 5
  3. 2025An Augmentation Overlap Theory of Contrastive Learningcited by 5
  4. 2024Referee Can Play: An Alternative Approach to Conditional Generation via Model Inversioncited by 4
  5. 2024Navigating the Effect of Parametrization for Dimensionality Reductioncited by 3
  6. 2025An Augmentation-Aware Theory for Self-Supervised Contrastive Learningcited by 3
  7. 2025Difficult Examples Hurt Unsupervised Contrastive Learning: A Theoretical Perspectivecited by 2
  8. 2024FedNE: Surrogate-Assisted Federated Neighbor Embedding for Dimensionality Reductioncited by 1
  9. 2025Node Embeddings via Neighbor Embeddingscited by 1
  10. 2026MAPLE: Self-Supervised Learning-Enhanced Nonlinear Dimensionality Reduction for Visual Analysiscited by 1
  11. 2024On the Generalization and Causal Explanation in Self-Supervised Learningcited by 0
  12. 2024Pseudo-Label Enhanced Prototypical Contrastive Learning for Uniformed Intent Discoverycited by 0
  13. 2026Group Contrastive Learning for Weakly Paired Multimodal Datacited by 0
  14. 2026UMAP Is Spectral Clustering on the Fuzzy Nearest-Neighbor Graphcited by 0
  15. 2026Maximizing Incremental Information Entropy for Contrastive Learningcited by 0
  16. 2026Say, Echo, Do: Strategic Narratives and Revealed Positioning in Financial Marketscited by 0