# Mixup Regularization: A Probabilistic Perspective

arXiv: https://arxiv.org/abs/2502.13825 · HTML: https://mlsanity.com/paper/2502.13825/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 38 references that are in the map.
- First posted on arXiv 19 Feb 2025.

## Builds on (most cited first, 21 of 21)

- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Mixtral of Experts](https://mlsanity.com/paper/2401.04088/) (2024)
- [GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding](https://mlsanity.com/paper/2006.16668/) (2020)
- [A Survey on Mixture of Experts in Large Language Models](https://mlsanity.com/paper/2407.06204/) (2024)
- [How Does Mixup Help With Robustness and Generalization?](https://mlsanity.com/paper/2010.04819/) (2020)
- [MixUp as Locally Linear Out-Of-Manifold Regularization](https://mlsanity.com/paper/1809.02499/) (2018)
- [G-Mixup: Graph Data Augmentation for Graph Classification](https://mlsanity.com/paper/2202.07179/) (2022)
- [Augmenting Data with Mixup for Sentence Classification: An Empirical Study](https://mlsanity.com/paper/1905.08941/) (2019)
- [On Mixup Regularization](https://mlsanity.com/paper/2006.06049/) (2020)
- [DiffuseMix: Label-Preserving Data Augmentation with Diffusion Models](https://mlsanity.com/paper/2405.14881/) (2024)
- [C-Mixup: Improving Generalization in Regression](https://mlsanity.com/paper/2210.05775/) (2022)
- [SeqMix: Augmenting Active Sequence Labeling via Sequence Mixup](https://mlsanity.com/paper/2010.02322/) (2020)
- [Between-class Learning for Image Classification](https://mlsanity.com/paper/1711.10284/) (2017)
- [MixMAE: Mixed and Masked Autoencoder for Efficient Pretraining of Hierarchical Vision Transformers](https://mlsanity.com/paper/2205.13137/) (2022)
- [Enhance Image Classification via Inter-Class Image Mixup with Diffusion Model](https://mlsanity.com/paper/2403.19600/) (2024)
- [Fusion of Probability Density Functions](https://mlsanity.com/paper/2202.11633/) (2022)
- [A Survey on Mixup Augmentations and Beyond](https://mlsanity.com/paper/2409.05202/) (2024)
- [Contrastive-mixup learning for improved speaker verification](https://mlsanity.com/paper/2202.10672/) (2022)
- [Deep Gaussian Mixture Ensembles](https://mlsanity.com/paper/2306.07235/) (2023)
- [Augment on Manifold: Mixup Regularization with UMAP](https://mlsanity.com/paper/2312.13141/) (2023)

## Built on by (most cited first, 1 of 1)

- [Beyond Mixtures and Products for Ensemble Aggregation: A Likelihood Perspective on Generalized Means](https://mlsanity.com/paper/2603.04204/) (2026)

## Abstract (arXiv)

In recent years, mixup regularization has gained popularity as an effective way to improve the generalization performance of deep learning models by training on convex combinations of training data. While many mixup variants have been explored, the proper adoption of the technique to conditional density estimation and probabilistic machine learning remains relatively unexplored. This work introduces a novel framework for mixup regularization based on probabilistic fusion that is better suited for conditional density estimation tasks. For data distributed according to a member of the exponential family, we show that likelihood functions can be analytically fused using log-linear pooling. We further propose an extension of probabilistic mixup, which allows for fusion of inputs at an arbitrary intermediate layer of the neural network. We provide a theoretical analysis comparing our approach to standard mixup variants. Empirical results on synthetic and real datasets demonstrate the benefits of our proposed framework compared to existing mixup variants.