# Linearly Convergent Mixup Learning

arXiv: https://arxiv.org/abs/2501.07794 · HTML: https://mlsanity.com/paper/2501.07794/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 28 references that are in the map.
- First posted on arXiv 14 Jan 2025.

## Builds on (most cited first, 11 of 11)

- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty](https://mlsanity.com/paper/1912.02781/) (2019)
- [How Does Mixup Help With Robustness and Generalization?](https://mlsanity.com/paper/2010.04819/) (2020)
- [On Mixup Regularization](https://mlsanity.com/paper/2006.06049/) (2020)
- [MixAugment & Mixup: Augmentation Methods for Facial Expression Recognition](https://mlsanity.com/paper/2205.04442/) (2022)
- [MixupE: Understanding and Improving Mixup from Directional Derivative Perspective](https://mlsanity.com/paper/2212.13381/) (2022)
- [Frank-Wolfe algorithm for learning SVM-type multi-category classifiers](https://mlsanity.com/paper/2008.08894/) (2020)
- [Scalable Dual Coordinate Descent for Kernel Methods](https://mlsanity.com/paper/2406.18001/) (2024)
- [A Comparison between Deep Neural Nets and Kernel Acoustic Models for Speech Recognition](https://mlsanity.com/paper/1603.05800/) (2016)
- [Learning Sign-Constrained Support Vector Machines](https://mlsanity.com/paper/2101.01473/) (2021)

## Abstract (arXiv)

Learning in the reproducing kernel Hilbert space (RKHS) such as the support vector machine has been recognized as a promising technique. It continues to be highly effective and competitive in numerous prediction tasks, particularly in settings where there is a shortage of training data or computational limitations exist. These methods are especially valued for their ability to work with small datasets and their interpretability. To address the issue of limited training data, mixup data augmentation, widely used in deep learning, has remained challenging to apply to learning in RKHS due to the generation of intermediate class labels. Although gradient descent methods handle these labels effectively, dual optimization approaches are typically not directly applicable. In this study, we present two novel algorithms that extend to a broader range of binary classification models. Unlike gradient-based approaches, our algorithms do not require hyperparameters like learning rates, simplifying their implementation and optimization. Both the number of iterations to converge and the computational cost per iteration scale linearly with respect to the dataset size. The numerical experiments demonstrate that our algorithms achieve faster convergence to the optimal solution compared to gradient descent approaches, and that mixup data augmentation consistently improves the predictive performance across various loss functions.