# Manifold Mixup: Better Representations by Interpolating Hidden States

arXiv: https://arxiv.org/abs/1806.05236 · HTML: https://mlsanity.com/paper/1806.05236/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Jun 2018.

## Built on by (most cited first, 14 of 14)

- [Prompt Learning via Meta-Regularization](https://mlsanity.com/paper/2404.00851/) (2024)
- [k-Mixup Regularization for Deep Learning via Optimal Transport](https://mlsanity.com/paper/2106.02933/) (2021)
- [Triple-stream Deep Metric Learning of Great Ape Behavioural Actions](https://mlsanity.com/paper/2301.02642/) (2023)
- [Decoupled Greedy Learning of CNNs for Synchronous and Asynchronous Distributed Learning](https://mlsanity.com/paper/2106.06401/) (2021)
- [GLAD: Generalizable Tuning for Vision-Language Models](https://mlsanity.com/paper/2507.13089/) (2025)
- [Revisiting Deep Semi-supervised Learning: An Empirical Distribution Alignment Framework and Its Generalization Bound](https://mlsanity.com/paper/2203.06639/) (2022)
- [Diffusion-Augmented Coreset Expansion for Scalable Dataset Distillation](https://mlsanity.com/paper/2412.04668/) (2024)
- [Dispersion Loss Counteracts Embedding Condensation and Improves Generalization in Small Language Models](https://mlsanity.com/paper/2602.00217/) (2026)
- [DNABERT-S: Pioneering Species Differentiation with Species-Aware DNA Embeddings](https://mlsanity.com/paper/2402.08777/) (2024)
- [InterLUDE: Interactions between Labeled and Unlabeled Data to Enhance Semi-Supervised Learning](https://mlsanity.com/paper/2403.10658/) (2024)
- [Mix from Failure: Confusion-Pairing Mixup for Long-Tailed Recognition](https://mlsanity.com/paper/2411.07621/) (2024)
- [Three Things to Know about Deep Metric Learning](https://mlsanity.com/paper/2412.12432/) (2024)
- [Boosting Open Set Recognition Performance through Modulated Representation Learning](https://mlsanity.com/paper/2505.18137/) (2025)
- [What Makes Synthetic Hard Negatives Work in Vision-Language Pretraining?](https://mlsanity.com/paper/2610.09700/) (2026)

## Abstract (arXiv)

Deep neural networks excel at learning the training data, but often provide incorrect and confident predictions when evaluated on slightly different test examples. This includes distribution shifts, outliers, and adversarial examples. To address these issues, we propose Manifold Mixup, a simple regularizer that encourages neural networks to predict less confidently on interpolations of hidden representations. Manifold Mixup leverages semantic interpolations as additional training signal, obtaining neural networks with smoother decision boundaries at multiple levels of representation. As a result, neural networks trained with Manifold Mixup learn class-representations with fewer directions of variance. We prove theory on why this flattening happens under ideal conditions, validate it on practical situations, and connect it to previous works on information theory and generalization. In spite of incurring no significant computation and being implemented in a few lines of code, Manifold Mixup improves strong baselines in supervised learning, robustness to single-step adversarial attacks, and test log-likelihood.