# i-Mix: A Domain-Agnostic Strategy for Contrastive Representation Learning

arXiv: https://arxiv.org/abs/2010.08887 · HTML: https://mlsanity.com/paper/2010.08887/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Oct 2020.

## Built on by (most cited first, 19 of 19)

- [An Empirical Study of Graph Contrastive Learning](https://mlsanity.com/paper/2109.01116/) (2021)
- [BYOL for Audio: Self-Supervised Learning for General-Purpose Audio Representation](https://mlsanity.com/paper/2103.06695/) (2021)
- [Supporting Clustering with Contrastive Learning](https://mlsanity.com/paper/2103.12953/) (2021)
- [Unleashing the Power of Contrastive Self-Supervised Visual Models via Contrast-Regularized Fine-Tuning](https://mlsanity.com/paper/2102.06605/) (2021)
- [Improving Contrastive Learning by Visualizing Feature Transformation](https://mlsanity.com/paper/2108.02982/) (2021)
- [MixMo: Mixing Multiple Inputs for Multiple Outputs via Deep Subnetworks](https://mlsanity.com/paper/2103.06132/) (2021)
- [Survey: Image Mixing and Deleting for Data Augmentation](https://mlsanity.com/paper/2106.07085/) (2021)
- [Towards Understanding the Data Dependency of Mixup-style Training](https://mlsanity.com/paper/2110.07647/) (2021)
- [Provably Learning Diverse Features in Multi-View Data with Midpoint Mixup](https://mlsanity.com/paper/2210.13512/) (2022)
- [Is your noise correction noisy? PLS: Robustness to label noise with two stage detection](https://mlsanity.com/paper/2210.04578/) (2022)
- [Hybrid Generative-Contrastive Representation Learning](https://mlsanity.com/paper/2106.06162/) (2021)
- [A Theory-Driven Self-Labeling Refinement Method for Contrastive Representation Learning](https://mlsanity.com/paper/2106.14749/) (2021)
- [A Low Rank Promoting Prior for Unsupervised Contrastive Learning](https://mlsanity.com/paper/2108.02696/) (2021)
- [Embedding contrastive unsupervised features to cluster in- and out-of-distribution noise in corrupted image datasets](https://mlsanity.com/paper/2207.01573/) (2022)
- [Data Augmentation by Selecting Mixed Classes Considering Distance Between Classes](https://mlsanity.com/paper/2209.05122/) (2022)
- [Unsupervised Representation Learning for Binary Networks by Joint Classifier Learning](https://mlsanity.com/paper/2110.08851/) (2021)
- [DNABERT-S: Pioneering Species Differentiation with Species-Aware DNA Embeddings](https://mlsanity.com/paper/2402.08777/) (2024)
- [An accurate detection is not all you need to combat label noise in web-noisy datasets](https://mlsanity.com/paper/2407.05528/) (2024)
- [Feature Augmentation for Self-supervised Contrastive Learning: A Closer Look](https://mlsanity.com/paper/2410.12396/) (2024)

## Abstract (arXiv)

Contrastive representation learning has shown to be effective to learn representations from unlabeled data. However, much progress has been made in vision domains relying on data augmentations carefully designed using domain knowledge. In this work, we propose i-Mix, a simple yet effective domain-agnostic regularization strategy for improving contrastive representation learning. We cast contrastive learning as training a non-parametric classifier by assigning a unique virtual class to each data in a batch. Then, data instances are mixed in both the input and virtual label spaces, providing more augmented data during training. In experiments, we demonstrate that i-Mix consistently improves the quality of learned representations across domains, including image, speech, and tabular data. Furthermore, we confirm its regularization effect via extensive ablation studies across model and dataset sizes. The code is available at https://github.com/kibok90/imix.