# A Note on Connecting Barlow Twins with Negative-Sample-Free Contrastive Learning

arXiv: https://arxiv.org/abs/2104.13712 · HTML: https://mlsanity.com/paper/2104.13712/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 16 references that are in the map.
- First posted on arXiv 28 Apr 2021.

## Builds on (most cited first, 9 of 9)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://mlsanity.com/paper/2103.03230/) (2021)
- [Learning deep representations by mutual information estimation and maximization](https://mlsanity.com/paper/1808.06670/) (2018)
- [Wasserstein Dependency Measure for Representation Learning](https://mlsanity.com/paper/1903.11780/) (2019)
- [Self-supervised Representation Learning with Relative Predictive Coding](https://mlsanity.com/paper/2103.11275/) (2021)

## Built on by (most cited first, 26 of 26)

- [An Empirical Study of Graph Contrastive Learning](https://mlsanity.com/paper/2109.01116/) (2021)
- [On the duality between contrastive and non-contrastive self-supervised learning](https://mlsanity.com/paper/2206.02574/) (2022)
- [Graph Barlow Twins: A self-supervised representation learning framework for graphs](https://mlsanity.com/paper/2106.02466/) (2021)
- [Self-Supervised Learning with Kernel Dependence Maximization](https://mlsanity.com/paper/2106.08320/) (2021)
- [COCOA: Cross Modality Contrastive Learning for Sensor Data](https://mlsanity.com/paper/2208.00467/) (2022)
- [Matrix Information Theory for Self-Supervised Learning](https://mlsanity.com/paper/2305.17326/) (2023)
- [Information Flow in Self-Supervised Learning](https://mlsanity.com/paper/2309.17281/) (2023)
- [Variance Covariance Regularization Enforces Pairwise Independence in Self-Supervised Representations](https://mlsanity.com/paper/2209.14905/) (2022)
- [RenyiCL: Contrastive Representation Learning with Skew Renyi Divergence](https://mlsanity.com/paper/2208.06270/) (2022)
- [Guarding Barlow Twins Against Overfitting with Mixed Samples](https://mlsanity.com/paper/2312.02151/) (2023)
- [DECAR: Deep Clustering for learning general-purpose Audio Representations](https://mlsanity.com/paper/2110.08895/) (2021)
- [PEER pressure: Model-to-Model Regularization for Single Source Domain Generalization](https://mlsanity.com/paper/2505.12745/) (2025)
- [Augmentation Component Analysis: Modeling Similarity via the Augmentation Overlaps](https://mlsanity.com/paper/2206.00471/) (2022)
- [Audio Barlow Twins: Self-Supervised Audio Representation Learning](https://mlsanity.com/paper/2209.14345/) (2022)
- [Beyond Pairwise Correlations: Higher-Order Redundancies in Self-Supervised Representation Learning](https://mlsanity.com/paper/2412.01926/) (2024)
- [FroSSL: Frobenius Norm Minimization for Efficient Multiview Self-Supervised Learning](https://mlsanity.com/paper/2310.02903/) (2023)
- [GraphVICRegHSIC: Towards improved self-supervised representation learning for graphs with a hyrbid loss function](https://mlsanity.com/paper/2105.12247/) (2021)
- [DeLoRes: Decorrelating Latent Spaces for Low-Resource Audio Representation Learning](https://mlsanity.com/paper/2203.13628/) (2022)
- [KG-NSF: Knowledge Graph Completion with a Negative-Sample-Free Approach](https://mlsanity.com/paper/2207.14617/) (2022)
- [Supervised Pretraining for Material Property Prediction](https://mlsanity.com/paper/2504.20112/) (2025)
- [Out-of-Distribution Detection with Hilbert-Schmidt Independence Optimization](https://mlsanity.com/paper/2209.12807/) (2022)
- [BTSeg: Barlow Twins Regularization for Domain Adaptation in Semantic Segmentation](https://mlsanity.com/paper/2308.16819/) (2023)
- [An Empirical Study of Self-supervised Learning with Wasserstein Distance](https://mlsanity.com/paper/2310.10143/) (2023)
- [Deep Tensor Network](https://mlsanity.com/paper/2311.11091/) (2023)
- [Maximum Manifold Capacity Representations in State Representation Learning](https://mlsanity.com/paper/2405.13848/) (2024)
- [Self-Supervised Representation Learning as Mutual Information Maximization](https://mlsanity.com/paper/2510.01345/) (2025)

## Abstract (arXiv)

In this report, we relate the algorithmic design of Barlow Twins' method to the Hilbert-Schmidt Independence Criterion (HSIC), thus establishing it as a contrastive learning approach that is free of negative samples. Through this perspective, we argue that Barlow Twins (and thus the class of negative-sample-free contrastive learning methods) suggests a possibility to bridge the two major families of self-supervised learning philosophies: non-contrastive and contrastive approaches. In particular, Barlow twins exemplified how we could combine the best practices of both worlds: avoiding the need of large training batch size and negative sample pairing (like non-contrastive methods) and avoiding symmetry-breaking network designs (like contrastive methods).