# SelfAugment: Automatic Augmentation Policies for Self-Supervised Learning

arXiv: https://arxiv.org/abs/2009.07724 · HTML: https://mlsanity.com/paper/2009.07724/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Sep 2020.

## Built on by (most cited first, 30 of 32)

- [A Cookbook of Self-Supervised Learning](https://mlsanity.com/paper/2304.12210/) (2023)
- [On Feature Decorrelation in Self-Supervised Learning](https://mlsanity.com/paper/2105.00470/) (2021)
- [RankMe: Assessing the downstream performance of pretrained self-supervised representations by their rank](https://mlsanity.com/paper/2210.02885/) (2022)
- [Predicting with Confidence on Unseen Distributions](https://mlsanity.com/paper/2107.03315/) (2021)
- [Chaos is a Ladder: A New Theoretical Understanding of Contrastive Learning via Augmentation Overlap](https://mlsanity.com/paper/2203.13457/) (2022)
- [Region Similarity Representation Learning](https://mlsanity.com/paper/2103.12902/) (2021)
- [Self-Supervised Pretraining Improves Self-Supervised Pretraining](https://mlsanity.com/paper/2103.12718/) (2021)
- [On the Importance and Applicability of Pre-Training for Federated Learning](https://mlsanity.com/paper/2206.11488/) (2022)
- [TeachAugment: Data Augmentation Optimization Using Teacher Knowledge](https://mlsanity.com/paper/2202.12513/) (2022)
- [No True State-of-the-Art? OOD Detection Methods are Inconsistent across Datasets](https://mlsanity.com/paper/2109.05554/) (2021)
- [On the Importance of Hyperparameters and Data Augmentation for Self-Supervised Learning](https://mlsanity.com/paper/2207.07875/) (2022)
- [A Survey of Automated Data Augmentation Algorithms for Deep Learning-based Image Classification Tasks](https://mlsanity.com/paper/2206.06544/) (2022)
- [Global Contrast Masked Autoencoders Are Powerful Pathological Representation Learners](https://mlsanity.com/paper/2205.09048/) (2022)
- [A Review of Predictive and Contrastive Self-supervised Learning for Medical Images](https://mlsanity.com/paper/2302.05043/) (2023)
- [RenyiCL: Contrastive Representation Learning with Skew Renyi Divergence](https://mlsanity.com/paper/2208.06270/) (2022)
- [Occam's Razor for Self Supervised Learning: What is Sufficient to Learn Good Representations?](https://mlsanity.com/paper/2406.10743/) (2024)
- [Constrained Mean Shift Using Distant Yet Related Neighbors for Representation Learning](https://mlsanity.com/paper/2112.04607/) (2021)
- [Prior Knowledge-Guided Attention in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2209.03745/) (2022)
- [No Free Lunch in Self Supervised Representation Learning](https://mlsanity.com/paper/2304.11718/) (2023)
- [Self-supervised Contrastive Learning for Irrigation Detection in Satellite Imagery](https://mlsanity.com/paper/2108.05484/) (2021)
- [Refine and Represent: Region-to-Object Representation Learning](https://mlsanity.com/paper/2208.11821/) (2022)
- [SynCo: Synthetic Hard Negatives for Contrastive Visual Representation Learning](https://mlsanity.com/paper/2410.02401/) (2024)
- [Unsupervised Learning on a DIET: Datum IndEx as Target Free of Self-Supervision, Reconstruction, Projector Head](https://mlsanity.com/paper/2302.10260/) (2023)
- [Which Augmentation Should I Use? An Empirical Investigation of Augmentations for Self-Supervised Phonocardiogram Representation Learning](https://mlsanity.com/paper/2312.00502/) (2023)
- [SASSL: Enhancing Self-Supervised Learning via Neural Style Transfer](https://mlsanity.com/paper/2312.01187/) (2023)
- [Contrastive Representation Learning with Trainable Augmentation Channel](https://mlsanity.com/paper/2111.07679/) (2021)
- [Improving Deepfake Detection with Reinforcement Learning-Based Adaptive Data Augmentation](https://mlsanity.com/paper/2511.07051/) (2025)
- [Multi-Augmentation for Efficient Visual Representation Learning for Self-supervised Pre-training](https://mlsanity.com/paper/2205.11772/) (2022)
- [Using Representation Expressiveness and Learnability to Evaluate Self-Supervised Learning Methods](https://mlsanity.com/paper/2206.01251/) (2022)
- [Self-supervised Learning for Anomaly Detection in Computational Workflows](https://mlsanity.com/paper/2310.01247/) (2023)

## Abstract (arXiv)

A common practice in unsupervised representation learning is to use labeled data to evaluate the quality of the learned representations. This supervised evaluation is then used to guide critical aspects of the training process such as selecting the data augmentation policy. However, guiding an unsupervised training process through supervised evaluations is not possible for real-world data that does not actually contain labels (which may be the case, for example, in privacy sensitive fields such as medical imaging). Therefore, in this work we show that evaluating the learned representations with a self-supervised image rotation task is highly correlated with a standard set of supervised evaluations (rank correlation > 0.94). We establish this correlation across hundreds of augmentation policies, training settings, and network architectures and provide an algorithm (SelfAugment) to automatically and efficiently select augmentation policies without using supervised evaluations. Despite not using any labeled data, the learned augmentation policies perform comparably with augmentation policies that were determined using exhaustive supervised evaluations.