mlsanityData to 7 Oct 2026

Revisiting Contrastive Learning through the Lens of Neighborhood Component Analysis: an Integrated Framework

arXiv 2112.04468 ↗First posted 8 Dec 2021cs.LG cs.CVcited by 11

Not on the map. Why?

Abstract (arXiv)

As a seminal tool in self-supervised representation learning, contrastive learning has gained unprecedented attention in recent years. In essence, contrastive learning aims to leverage pairs of positive and negative samples for representation learning, which relates to exploiting neighborhood information in a feature space. By investigating the connection between contrastive learning and neighborhood component analysis (NCA), we provide a novel stochastic nearest neighbor viewpoint of contrastive learning and subsequently propose a series of contrastive losses that outperform the existing ones. Under our proposed framework, we show a new methodology to design integrated contrastive losses that could simultaneously achieve good accuracy and robustness on downstream tasks. With the integrated framework, we achieve up to 6% improvement on the standard accuracy and 17% improvement on the robust accuracy.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 33 of its 46 references that are in the map. First posted on arXiv 8 Dec 2021.

Builds on

33 of 46 references.

References · 33

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  3. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  4. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  5. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  6. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  7. 2013Intriguing properties of neural networkscited by 5,114
  8. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  9. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  10. 2020Supervised Contrastive Learningcited by 2,803
  11. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  12. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
21 more
  1. 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,315
  2. 2019Contrastive Multiview Codingcited by 1,297
  3. 2016Adversarial Machine Learning at Scalecited by 934
  4. 2020What Makes for Good Views for Contrastive Learning?cited by 843
  5. 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 774
  6. 2020Fast is better than free: Revisiting adversarial trainingcited by 723
  7. 2020Contrastive Multi-View Representation Learning on Graphscited by 651
  8. 2020Contrastive Learning with Hard Negative Samplescited by 619
  9. 2019A Theoretical Analysis of Contrastive Unsupervised Representation Learningcited by 530
  10. 2020Debiased Contrastive Learningcited by 354
  11. 2021With a Little Help from My Friends: Nearest-Neighbor Contrastive Learning of Visual Representationscited by 324
  12. 2021Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Losscited by 311
  13. 2019Unsupervised Embedding Learning via Invariant and Spreading Instance Featurecited by 231
  14. 2018Adversarial Logit Pairingcited by 224
  15. 2016Exploring the Limits of Language Modelingcited by 172
  16. 2020Adversarial Self-Supervised Contrastive Learningcited by 138
  17. 2017Unsupervised Learning by Predicting Noisecited by 97
  18. 2020Contrastive Learning with Adversarial Examplescited by 91
  19. 2021When Does Contrastive Learning Preserve Adversarial Robustness from Pretraining to Finetuning?cited by 76
  20. 2020Towards Domain-Agnostic Contrastive Learningcited by 75
  21. 2020Are Adversarial Examples Created Equal? A Learnable Weighted Minimax Risk for Robustness under Non-uniform Attackscited by 14

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 6; 2024: 0; 2025: 3.

11 papers.

  1. 2023Understanding Multimodal Contrastive Learning and Incorporating Unpaired Datacited by 43
  2. 2023The Trade-off between Universality and Label Efficiency of Representations from Contrastive Learningcited by 27
  3. 2023Understanding Contrastive Learning via Distributionally Robust Optimizationcited by 21
  4. 2023Rethinking Weak Supervision in Helping Contrastive Learningcited by 18
  5. 2022Hard Negative Sampling Strategies for Contrastive Representation Learningcited by 13
  6. 2023ConCerNet: A Contrastive Learning Based Framework for Automated Conservation Law Discovery and Trustworthy Dynamical System Predictioncited by 5
  7. 2023Towards the Sparseness of Projection Head in Self-Supervised Learningcited by 4
  8. 2025An Augmentation-Aware Theory for Self-Supervised Contrastive Learningcited by 3
  9. 2022Self-supervised Representation Learning on Electronic Health Records with Graph Kernel Infomaxcited by 2
  10. 2025Difficult Examples Hurt Unsupervised Contrastive Learning: A Theoretical Perspectivecited by 2
  11. 2025Revisiting Theory of Contrastive Learning for Domain Generalizationcited by 0