# SUMix: Mixup with Semantic and Uncertain Information

arXiv: https://arxiv.org/abs/2407.07805 · HTML: https://mlsanity.com/paper/2407.07805/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 59 of its 75 references that are in the map.
- First posted on arXiv 10 Jul 2024.

## Builds on (most cited first, 30 of 59)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [RandAugment: Practical automated data augmentation with a reduced search space](https://mlsanity.com/paper/1909.13719/) (2019)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [Fine-Grained Visual Classification of Aircraft](https://mlsanity.com/paper/1306.5151/) (2013)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [Improved Regularization of Convolutional Neural Networks with Cutout](https://mlsanity.com/paper/1708.04552/) (2017)
- [AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty](https://mlsanity.com/paper/1912.02781/) (2019)
- [Show and Tell: A Neural Image Caption Generator](https://mlsanity.com/paper/1411.4555/) (2014)
- [AutoAugment: Learning Augmentation Policies from Data](https://mlsanity.com/paper/1805.09501/) (2018)
- [Intriguing Properties of Vision Transformers](https://mlsanity.com/paper/2105.10497/) (2021)
- [A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasets](https://mlsanity.com/paper/1707.08819/) (2017)
- [Puzzle Mix: Exploiting Saliency and Local Statistics for Optimal Mixup](https://mlsanity.com/paper/2009.06962/) (2020)
- [Fast AutoAugment](https://mlsanity.com/paper/1905.00397/) (2019)
- [Probabilistic Face Embeddings](https://mlsanity.com/paper/1904.09658/) (2019)

## Built on by (most cited first, 12 of 12)

- [A Survey on Mixup Augmentations and Beyond](https://mlsanity.com/paper/2409.05202/) (2024)
- [Context-guided Responsible Data Augmentation with Diffusion Models](https://mlsanity.com/paper/2503.10687/) (2025)
- [MergeMix: A Unified Augmentation Paradigm for Visual and Multi-Modal Understanding](https://mlsanity.com/paper/2510.23479/) (2025)
- [SGD-Mix: Enhancing Domain-Specific Image Classification with Label-Preserving Data Augmentation](https://mlsanity.com/paper/2505.11813/) (2025)
- [Neural Architecture Search based Global-local Vision Mamba for Palm-Vein Recognition](https://mlsanity.com/paper/2408.05743/) (2024)
- [S2-FracMix: Label-Preserving Self-Saliency Mixup Augmentation](https://mlsanity.com/paper/2606.25784/) (2026)
- [EM-DARTS: Hierarchical Differentiable Architecture Search for Eye Movement Recognition](https://mlsanity.com/paper/2409.14432/) (2024)
- [Compositional Attribute Imbalance in Vision Datasets](https://mlsanity.com/paper/2506.14418/) (2025)
- [Enhancing Ambiguous Dynamic Facial Expression Recognition with Soft Label-based Data Augmentation](https://mlsanity.com/paper/2506.20867/) (2025)
- [MS-Mix: Sentiment-Guided Adaptive Augmentation for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2510.11579/) (2025)
- [ZScribbleSeg: A comprehensive segmentation framework with modeling of efficient annotation and maximization of scribble supervision](https://mlsanity.com/paper/2605.06266/) (2026)
- [DisMix: Order-Aware Mixup for Medical Imaging via Disentangling Ordinal and Non-Ordinal Features](https://mlsanity.com/paper/2608.04652/) (2026)

## Abstract (arXiv)

Mixup data augmentation approaches have been applied for various tasks of deep learning to improve the generalization ability of deep neural networks. Some existing approaches CutMix, SaliencyMix, etc. randomly replace a patch in one image with patches from another to generate the mixed image. Similarly, the corresponding labels are linearly combined by a fixed ratio λ by l. The objects in two images may be overlapped during the mixing process, so some semantic information is corrupted in the mixed samples. In this case, the mixed image does not match the mixed label information. Besides, such a label may mislead the deep learning model training, which results in poor performance. To solve this problem, we proposed a novel approach named SUMix to learn the mixing ratio as well as the uncertainty for the mixed samples during the training process. First, we design a learnable similarity function to compute an accurate mix ratio. Second, an approach is investigated as a regularized term to model the uncertainty of the mixed samples. We conduct experiments on five image benchmarks, and extensive experimental results imply that our method is capable of improving the performance of classifiers with different cutting-based mixup approaches. The source code is available at https://github.com/JinXins/SUMix.