# An Augmentation-Aware Theory for Self-Supervised Contrastive Learning

arXiv: https://arxiv.org/abs/2505.22196 · HTML: https://mlsanity.com/paper/2505.22196/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 31 references that are in the map.
- First posted on arXiv 28 May 2025.

## Builds on (most cited first, 29 of 29)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://mlsanity.com/paper/2103.03230/) (2021)
- [An Empirical Study of Training Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.02057/) (2021)
- [Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere](https://mlsanity.com/paper/2005.10242/) (2020)
- [What Makes for Good Views for Contrastive Learning?](https://mlsanity.com/paper/2005.10243/) (2020)
- [A Theoretical Analysis of Contrastive Unsupervised Representation Learning](https://mlsanity.com/paper/1902.09229/) (2019)
- [Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss](https://mlsanity.com/paper/2106.04156/) (2021)
- [Contrastive Learning Inverts the Data Generating Process](https://mlsanity.com/paper/2102.08850/) (2021)
- [Chaos is a Ladder: A New Theoretical Understanding of Contrastive Learning via Augmentation Overlap](https://mlsanity.com/paper/2203.13457/) (2022)
- [Towards the Generalization of Contrastive Self-Supervised Learning](https://mlsanity.com/paper/2111.00743/) (2021)
- [Sequence-to-Sequence Contrastive Learning for Text Recognition](https://mlsanity.com/paper/2012.10873/) (2020)
- [Connect, Not Collapse: Explaining Contrastive Learning for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2204.00570/) (2022)
- [On Mutual Information in Contrastive Learning for Visual Representations](https://mlsanity.com/paper/2005.13149/) (2020)
- [Investigating the Role of Negatives in Contrastive Representation Learning](https://mlsanity.com/paper/2106.09943/) (2021)
- [Understanding Negative Samples in Instance Discriminative Self-supervised Representation Learning](https://mlsanity.com/paper/2102.06866/) (2021)
- [On the Generalization of Multi-modal Contrastive Learning](https://mlsanity.com/paper/2306.04272/) (2023)
- [Your Contrastive Learning Is Secretly Doing Stochastic Neighbor Embedding](https://mlsanity.com/paper/2205.14814/) (2022)
- [On the Surrogate Gap between Contrastive and Supervised Losses](https://mlsanity.com/paper/2110.02501/) (2021)
- [Generalization Analysis for Contrastive Representation Learning](https://mlsanity.com/paper/2302.12383/) (2023)
- [A Message Passing Perspective on Learning Dynamics of Contrastive Learning](https://mlsanity.com/paper/2303.04435/) (2023)
- [Understanding Contrastive Learning via Distributionally Robust Optimization](https://mlsanity.com/paper/2310.11048/) (2023)
- [Non-negative Contrastive Learning](https://mlsanity.com/paper/2403.12459/) (2024)
- [Rethinking Weak Supervision in Helping Contrastive Learning](https://mlsanity.com/paper/2306.04160/) (2023)
- [Revisiting Contrastive Learning through the Lens of Neighborhood Component Analysis: an Integrated Framework](https://mlsanity.com/paper/2112.04468/) (2021)
- [Learning to Discriminate Information for Online Action Detection: Analysis and Application](https://mlsanity.com/paper/2109.03393/) (2021)

## Built on by (most cited first, 3 of 3)

- [An Augmentation Overlap Theory of Contrastive Learning](https://mlsanity.com/paper/2511.03114/) (2025)
- [Image Augmentation as Test Generation for Deep Learning-Based Image Retrieval Systems](https://mlsanity.com/paper/2608.27502/) (2026)
- [A Theoretical Framework for Masked Pretraining (MPT)](https://mlsanity.com/paper/2609.06460/) (2026)

## Abstract (arXiv)

Self-supervised contrastive learning has emerged as a powerful tool in machine learning and computer vision to learn meaningful representations from unlabeled data. Meanwhile, its empirical success has encouraged many theoretical studies to reveal the learning mechanisms. However, in the existing theoretical research, the role of data augmentation is still under-exploited, especially the effects of specific augmentation types. To fill in the blank, we for the first time propose an augmentation-aware error bound for self-supervised contrastive learning, showing that the supervised risk is bounded not only by the unsupervised risk, but also explicitly by a trade-off induced by data augmentation. Then, under a novel semantic label assumption, we discuss how certain augmentation methods affect the error bound. Lastly, we conduct both pixel- and representation-level experiments to verify our proposed theoretical results.