# Batch Curation for Unsupervised Contrastive Representation Learning

arXiv: https://arxiv.org/abs/2108.08643 · HTML: https://mlsanity.com/paper/2108.08643/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 11 references that are in the map.
- First posted on arXiv 19 Aug 2021.

## Builds on (most cited first, 8 of 8)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1603.09246/) (2016)
- [Self-supervised Pretraining of Visual Features in the Wild](https://mlsanity.com/paper/2103.01988/) (2021)
- [Learning Features by Watching Objects Move](https://mlsanity.com/paper/1612.06370/) (2016)

## Built on by (most cited first, 1 of 1)

- [The Bad Batches: Enhancing Self-Supervised Learning in Image Classification Through Representative Batch Curation](https://mlsanity.com/paper/2403.19579/) (2024)

## Abstract (arXiv)

The state-of-the-art unsupervised contrastive visual representation learning methods that have emerged recently (SimCLR, MoCo, SwAV) all make use of data augmentations in order to construct a pretext task of instant discrimination consisting of similar and dissimilar pairs of images. Similar pairs are constructed by randomly extracting patches from the same image and applying several other transformations such as color jittering or blurring, while transformed patches from different image instances in a given batch are regarded as dissimilar pairs. We argue that this approach can result similar pairs that are textitsemantically dissimilar. In this work, we address this problem by introducing a textitbatch curation scheme that selects batches during the training process that are more inline with the underlying contrastive objective. We provide insights into what constitutes beneficial similar and dissimilar pairs as well as validate textitbatch curation on CIFAR10 by integrating it in the SimCLR model.