mlsanityData to 14 Oct 2026

A Note on Connecting Barlow Twins with Negative-Sample-Free Contrastive Learning

arXiv 2104.13712 ↗First posted 28 Apr 2021cs.LGcited by 26

Not on the map. Why?

Abstract (arXiv)

In this report, we relate the algorithmic design of Barlow Twins' method to the Hilbert-Schmidt Independence Criterion (HSIC), thus establishing it as a contrastive learning approach that is free of negative samples. Through this perspective, we argue that Barlow Twins (and thus the class of negative-sample-free contrastive learning methods) suggests a possibility to bridge the two major families of self-supervised learning philosophies: non-contrastive and contrastive approaches. In particular, Barlow twins exemplified how we could combine the best practices of both worlds: avoiding the need of large training batch size and negative sample pairing (like non-contrastive methods) and avoiding symmetry-breaking network designs (like contrastive methods).

Key facts. Cited by 26 papers in the map (data through 14 Oct 2026). Builds on 9 of its 16 references that are in the map. First posted on arXiv 28 Apr 2021.

Builds on

9 of 16 references.

References · 9

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  3. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  4. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  5. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  6. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,838
  7. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,273
  8. 2019Wasserstein Dependency Measure for Representation Learningcited by 69
  9. 2021Self-supervised Representation Learning with Relative Predictive Codingcited by 28

Built on by

Cited by year

Citing papers per year: 2021: 5; 2022: 9; 2023: 7; 2024: 2; 2025: 3.

26 papers.

Sort:

  1. 2021An Empirical Study of Graph Contrastive Learningcited by 113
  2. 2022On the duality between contrastive and non-contrastive self-supervised learningcited by 108
  3. 2021Graph Barlow Twins: A self-supervised representation learning framework for graphscited by 85
  4. 2021Self-Supervised Learning with Kernel Dependence Maximizationcited by 77
  5. 2022COCOA: Cross Modality Contrastive Learning for Sensor Datacited by 21
  6. 2023Matrix Information Theory for Self-Supervised Learningcited by 21
  7. 2023Information Flow in Self-Supervised Learningcited by 16
  8. 2022Variance Covariance Regularization Enforces Pairwise Independence in Self-Supervised Representationscited by 10
  9. 2022RenyiCL: Contrastive Representation Learning with Skew Renyi Divergencecited by 8
  10. 2023Guarding Barlow Twins Against Overfitting with Mixed Samplescited by 8
  11. 2021DECAR: Deep Clustering for learning general-purpose Audio Representationscited by 7
  12. 2025PEER pressure: Model-to-Model Regularization for Single Source Domain Generalizationcited by 7
14 more
  1. 2022Augmentation Component Analysis: Modeling Similarity via the Augmentation Overlapscited by 5
  2. 2022Audio Barlow Twins: Self-Supervised Audio Representation Learningcited by 5
  3. 2024Beyond Pairwise Correlations: Higher-Order Redundancies in Self-Supervised Representation Learningcited by 4
  4. 2023FroSSL: Frobenius Norm Minimization for Efficient Multiview Self-Supervised Learningcited by 2
  5. 2021GraphVICRegHSIC: Towards improved self-supervised representation learning for graphs with a hyrbid loss functioncited by 1
  6. 2022DeLoRes: Decorrelating Latent Spaces for Low-Resource Audio Representation Learningcited by 1
  7. 2022KG-NSF: Knowledge Graph Completion with a Negative-Sample-Free Approachcited by 1
  8. 2025Supervised Pretraining for Material Property Predictioncited by 1
  9. 2022Out-of-Distribution Detection with Hilbert-Schmidt Independence Optimizationcited by 0
  10. 2023BTSeg: Barlow Twins Regularization for Domain Adaptation in Semantic Segmentationcited by 0
  11. 2023An Empirical Study of Self-supervised Learning with Wasserstein Distancecited by 0
  12. 2023Deep Tensor Networkcited by 0
  13. 2024Maximum Manifold Capacity Representations in State Representation Learningcited by 0
  14. 2025Self-Supervised Representation Learning as Mutual Information Maximizationcited by 0