# Infinite Class Mixup

arXiv: https://arxiv.org/abs/2305.10293 · HTML: https://mlsanity.com/paper/2305.10293/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 50 of its 71 references that are in the map.
- First posted on arXiv 17 May 2023.

## Builds on (most cited first, 30 of 50)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [MixMatch: A Holistic Approach to Semi-Supervised Learning](https://mlsanity.com/paper/1905.02249/) (2019)
- [Class-Balanced Loss Based on Effective Number of Samples](https://mlsanity.com/paper/1901.05555/) (2019)
- [AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty](https://mlsanity.com/paper/1912.02781/) (2019)
- [On Mixup Training: Improved Calibration and Predictive Uncertainty for Deep Neural Networks](https://mlsanity.com/paper/1905.11001/) (2019)
- [Puzzle Mix: Exploiting Saliency and Local Statistics for Optimal Mixup](https://mlsanity.com/paper/2009.06962/) (2020)
- [Adversarial Domain Adaptation with Domain Mixup](https://mlsanity.com/paper/1912.01805/) (2019)
- [MixUp as Locally Linear Out-Of-Manifold Regularization](https://mlsanity.com/paper/1809.02499/) (2018)
- [Remix: Rebalanced Mixup](https://mlsanity.com/paper/2007.03943/) (2020)
- [On Translation Invariance in CNNs: Convolutional Layers can Exploit Absolute Spatial Location](https://mlsanity.com/paper/2003.07064/) (2020)
- [Charting the Right Manifold: Manifold Mixup for Few-shot Learning](https://mlsanity.com/paper/1907.12087/) (2019)
- [Co-Mixup: Saliency Guided Joint Mixup with Supermodular Diversity](https://mlsanity.com/paper/2102.03065/) (2021)
- [Fair Mixup: Fairness via Interpolation](https://mlsanity.com/paper/2103.06503/) (2021)
- [PointMixup: Augmentation for Point Clouds](https://mlsanity.com/paper/2008.06374/) (2020)
- [Context-Aware Mixup for Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2108.03557/) (2021)
- [TransMix: Attend to Mix for Vision Transformers](https://mlsanity.com/paper/2111.09833/) (2021)
- [PolarMix: A General Data Augmentation Technique for LiDAR Point Clouds](https://mlsanity.com/paper/2208.00223/) (2022)
- [Adversarial Vertex Mixup: Toward Better Adversarially Robust Generalization](https://mlsanity.com/paper/2003.02484/) (2020)
- [Vision-Language Navigation with Random Environmental Mixup](https://mlsanity.com/paper/2106.07876/) (2021)
- [PointCutMix: Regularization Strategy for Point Cloud Classification](https://mlsanity.com/paper/2101.01461/) (2021)
- [AutoMix: Unveiling the Power of Mixup for Stronger Classifiers](https://mlsanity.com/paper/2103.13027/) (2021)
- [Do We Train on Test Data? Purging CIFAR of Near-Duplicates](https://mlsanity.com/paper/1902.00423/) (2019)
- [Regularization Strategy for Point Cloud via Rigidly Mixed Sample](https://mlsanity.com/paper/2102.01929/) (2021)
- [C-Mixup: Improving Generalization in Regression](https://mlsanity.com/paper/2210.05775/) (2022)

## Built on by (most cited first, 2 of 2)

- [DiffuseMix: Label-Preserving Data Augmentation with Diffusion Models](https://mlsanity.com/paper/2405.14881/) (2024)
- [GenMix: Effective Data Augmentation with Generative Diffusion Model Image Editing](https://mlsanity.com/paper/2412.02366/) (2024)

## Abstract (arXiv)

Mixup is a widely adopted strategy for training deep networks, where additional samples are augmented by interpolating inputs and labels of training pairs. Mixup has shown to improve classification performance, network calibration, and out-of-distribution generalisation. While effective, a cornerstone of Mixup, namely that networks learn linear behaviour patterns between classes, is only indirectly enforced since the output interpolation is performed at the probability level. This paper seeks to address this limitation by mixing the classifiers directly instead of mixing the labels for each mixed pair. We propose to define the target of each augmented sample as a uniquely new classifier, whose parameters are a linear interpolation of the classifier vectors of the input pair. The space of all possible classifiers is continuous and spans all interpolations between classifier pairs. To make optimisation tractable, we propose a dual-contrastive Infinite Class Mixup loss, where we contrast the classifier of a mixed pair to both the classifiers and the predicted outputs of other mixed pairs in a batch. Infinite Class Mixup is generic in nature and applies to many variants of Mixup. Empirically, we show that it outperforms standard Mixup and variants such as RegMixup and Remix on balanced, long-tailed, and data-constrained benchmarks, highlighting its broad applicability.