# Contrastive Learning Is Spectral Clustering On Similarity Graph

arXiv: https://arxiv.org/abs/2303.15103 · HTML: https://mlsanity.com/paper/2303.15103/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 42 references that are in the map.
- First posted on arXiv 27 Mar 2023.

## Builds on (most cited first, 24 of 24)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere](https://mlsanity.com/paper/2005.10242/) (2020)
- [Graph Contrastive Learning with Augmentations](https://mlsanity.com/paper/2010.13902/) (2020)
- [What Makes for Good Views for Contrastive Learning?](https://mlsanity.com/paper/2005.10243/) (2020)
- [Contrastive Multi-View Representation Learning on Graphs](https://mlsanity.com/paper/2006.05582/) (2020)
- [A Theoretical Analysis of Contrastive Unsupervised Representation Learning](https://mlsanity.com/paper/1902.09229/) (2019)
- [Hard Negative Mixing for Contrastive Learning](https://mlsanity.com/paper/2010.01028/) (2020)
- [Tune: A Research Platform for Distributed Model Selection and Training](https://mlsanity.com/paper/1807.05118/) (2018)
- [Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss](https://mlsanity.com/paper/2106.04156/) (2021)
- [Parametric Contrastive Learning](https://mlsanity.com/paper/2107.12028/) (2021)
- [Contrastive and Non-Contrastive Self-Supervised Learning Recover Global and Local Spectral Embedding Methods](https://mlsanity.com/paper/2205.11508/) (2022)
- [Chaos is a Ladder: A New Theoretical Understanding of Contrastive Learning via Augmentation Overlap](https://mlsanity.com/paper/2203.13457/) (2022)
- [Deep Neural Tangent Kernel and Laplace Kernel Have the Same RKHS](https://mlsanity.com/paper/2009.10683/) (2020)
- [Understanding Deep Contrastive Learning via Coordinate-wise Optimization](https://mlsanity.com/paper/2201.12680/) (2022)
- [Your Contrastive Learning Is Secretly Doing Stochastic Neighbor Embedding](https://mlsanity.com/paper/2205.14814/) (2022)
- [Neural Eigenfunctions Are Structured Representation Learners](https://mlsanity.com/paper/2210.12637/) (2022)
- [A Probabilistic Graph Coupling View of Dimension Reduction](https://mlsanity.com/paper/2201.13053/) (2022)

## Built on by (most cited first, 25 of 25)

- [Matrix Information Theory for Self-Supervised Learning](https://mlsanity.com/paper/2305.17326/) (2023)
- [Non-negative Contrastive Learning](https://mlsanity.com/paper/2403.12459/) (2024)
- [Information Flow in Self-Supervised Learning](https://mlsanity.com/paper/2309.17281/) (2023)
- [Provable Contrastive Continual Learning](https://mlsanity.com/paper/2405.18756/) (2024)
- [Revisiting Self-Supervised Heterogeneous Graph Learning from Spectral Clustering Perspective](https://mlsanity.com/paper/2412.00742/) (2024)
- [OTMatch: Improving Semi-Supervised Learning with Optimal Transport](https://mlsanity.com/paper/2310.17455/) (2023)
- [Tight PAC-Bayesian Risk Certificates for Contrastive Learning](https://mlsanity.com/paper/2412.03486/) (2024)
- [THESAURUS: Contrastive Graph Clustering by Swapping Fused Gromov-Wasserstein Couplings](https://mlsanity.com/paper/2412.11550/) (2024)
- [Existing LLMs Are Not Self-Consistent For Simple Tasks](https://mlsanity.com/paper/2506.18781/) (2025)
- [LeakyCLIP: Extracting Training Data from CLIP](https://mlsanity.com/paper/2508.00756/) (2025)
- [PCA++: How Uniformity Induces Robustness to Background Noise in Contrastive Learning](https://mlsanity.com/paper/2511.12278/) (2025)
- [Lambda: Learning Matchable Prior For Entity Alignment with Unlabeled Dangling Cases](https://mlsanity.com/paper/2403.10978/) (2024)
- [COSTAR: Improved Temporal Counterfactual Estimation with Self-Supervised Learning](https://mlsanity.com/paper/2311.00886/) (2023)
- [Difficult Examples Hurt Unsupervised Contrastive Learning: A Theoretical Perspective](https://mlsanity.com/paper/2501.01317/) (2025)
- [A Markov Categorical Framework for Language Modeling](https://mlsanity.com/paper/2507.19247/) (2025)
- [Deep Tensor Network](https://mlsanity.com/paper/2311.11091/) (2023)
- [Enhancing Graph Self-Supervised Learning with Graph Interplay](https://mlsanity.com/paper/2410.04061/) (2024)
- [Affect-aware Cross-Domain Recommendation for Art Therapy via Music Preference Elicitation](https://mlsanity.com/paper/2507.21120/) (2025)
- [Enhancing Contrastive Link Prediction With Edge Balancing Augmentation](https://mlsanity.com/paper/2508.14808/) (2025)
- [Understanding InfoNCE: Transition Probability Matrix Induced Feature Clustering](https://mlsanity.com/paper/2511.12180/) (2025)
- [Graph Contrastive Learning via Spectral Graph Alignment](https://mlsanity.com/paper/2512.07878/) (2025)
- [Maximizing Incremental Information Entropy for Contrastive Learning](https://mlsanity.com/paper/2603.12594/) (2026)
- [Expressivity In Multimodal Contrastive Learning](https://mlsanity.com/paper/2608.17203/) (2026)
- [Refining Heuristic-Based Bitcoin Address Clustering with Graph Neural Networks](https://mlsanity.com/paper/2609.01942/) (2026)
- [Differentiable Koopman Operator for Contrastive Learning on Dynamic Graphs](https://mlsanity.com/paper/2610.02990/) (2026)

## Abstract (arXiv)

Contrastive learning is a powerful self-supervised learning method, but we have a limited theoretical understanding of how it works and why it works. In this paper, we prove that contrastive learning with the standard InfoNCE loss is equivalent to spectral clustering on the similarity graph. Using this equivalence as the building block, we extend our analysis to the CLIP model and rigorously characterize how similar multi-modal objects are embedded together. Motivated by our theoretical insights, we introduce the Kernel-InfoNCE loss, incorporating mixtures of kernel functions that outperform the standard Gaussian kernel on several vision datasets. The code is available at https://github.com/yifanzhang-pro/Kernel-InfoNCE.