# Noisy Feature Mixup

arXiv: https://arxiv.org/abs/2110.02180 · HTML: https://mlsanity.com/paper/2110.02180/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 51 of its 82 references that are in the map.
- First posted on arXiv 5 Oct 2021.

## Builds on (most cited first, 30 of 51)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [Theoretically Principled Trade-off between Robustness and Accuracy](https://mlsanity.com/paper/1901.08573/) (2019)
- [AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty](https://mlsanity.com/paper/1912.02781/) (2019)
- [Robustness May Be at Odds with Accuracy](https://mlsanity.com/paper/1805.12152/) (2018)
- [Overfitting in adversarially robust deep learning](https://mlsanity.com/paper/2002.11569/) (2020)
- [Certified Robustness to Adversarial Examples with Differential Privacy](https://mlsanity.com/paper/1802.03471/) (2018)
- [Adversarially Robust Generalization Requires More Data](https://mlsanity.com/paper/1804.11285/) (2018)
- [On Mixup Training: Improved Calibration and Predictive Uncertainty for Deep Neural Networks](https://mlsanity.com/paper/1905.11001/) (2019)
- [Escaping the Big Data Paradigm with Compact Transformers](https://mlsanity.com/paper/2104.05704/) (2021)
- [Puzzle Mix: Exploiting Saliency and Local Statistics for Optimal Mixup](https://mlsanity.com/paper/2009.06962/) (2020)
- [Vision Transformers are Robust Learners](https://mlsanity.com/paper/2105.07581/) (2021)
- [Fixing Data Augmentation to Improve Adversarial Robustness](https://mlsanity.com/paper/2103.01946/) (2021)
- [How Does Mixup Help With Robustness and Generalization?](https://mlsanity.com/paper/2010.04819/) (2020)
- [Sensitivity and Generalization in Neural Networks: an Empirical Study](https://mlsanity.com/paper/1802.08760/) (2018)
- [Formal Guarantees on the Robustness of a Classifier against Adversarial Manipulation](https://mlsanity.com/paper/1705.08475/) (2017)
- [Robustness via curvature regularization, and vice versa](https://mlsanity.com/paper/1811.09716/) (2018)
- [Wasserstein Distributionally Robust Optimization: Theory and Applications in Machine Learning](https://mlsanity.com/paper/1908.08729/) (2019)
- [Understanding and Mitigating the Tradeoff Between Robustness and Accuracy](https://mlsanity.com/paper/2002.10716/) (2020)
- [Is Robustness the Cost of Accuracy? -- A Comprehensive Study on the Robustness of 18 Deep Image Classification Models](https://mlsanity.com/paper/1808.01688/) (2018)
- [Robust Learning with Jacobian Regularization](https://mlsanity.com/paper/1908.02729/) (2019)
- [Robust Large Margin Deep Neural Networks](https://mlsanity.com/paper/1605.08254/) (2016)
- [Predicting the Generalization Gap in Deep Networks with Margin Distributions](https://mlsanity.com/paper/1810.00113/) (2018)
- [Large Margin Deep Networks for Classification](https://mlsanity.com/paper/1803.05598/) (2018)

## Built on by (most cited first, 27 of 27)

- [A Survey of Mix-based Data Augmentation: Taxonomy, Methods, Applications, and Explainability](https://mlsanity.com/paper/2212.10888/) (2022)
- [FedFA: Federated Feature Augmentation](https://mlsanity.com/paper/2301.12995/) (2023)
- [DART: Diversify-Aggregate-Repeat Training Improves Generalization of Neural Networks](https://mlsanity.com/paper/2302.14685/) (2023)
- [NoisyMix: Boosting Model Robustness to Common Corruptions](https://mlsanity.com/paper/2202.01263/) (2022)
- [Concept Pinpoint Eraser for Text-to-image Diffusion Models via Residual Attention Gate](https://mlsanity.com/paper/2506.22806/) (2025)
- [First-Order Manifold Data Augmentation for Regression Learning](https://mlsanity.com/paper/2406.10914/) (2024)
- [Gated Recurrent Neural Networks with Weighted Time-Delay Feedback](https://mlsanity.com/paper/2212.00228/) (2022)
- [Tailoring Mixup to Data for Calibration](https://mlsanity.com/paper/2311.01434/) (2023)
- [Robust Asymmetric Heterogeneous Federated Learning with Corrupted Clients](https://mlsanity.com/paper/2503.09206/) (2025)
- [Investigating the Corruption Robustness of Image Classifiers with Random Lp-norm Corruptions](https://mlsanity.com/paper/2305.05400/) (2023)
- [Learning to Detour: Shortcut Mitigating Augmentation for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2405.18148/) (2024)
- [PDE+: Enhancing Generalization via PDE with Adaptive Distributional Diffusion](https://mlsanity.com/paper/2305.15835/) (2023)
- [For Better or For Worse? Learning Minimum Variance Features With Label Augmentation](https://mlsanity.com/paper/2402.06855/) (2024)
- [A Unified Deep Semantic Expansion Framework for Domain-Generalized Person Re-identification](https://mlsanity.com/paper/2410.08456/) (2024)
- [Memory Classifiers: Two-stage Classification for Robustness in Machine Learning](https://mlsanity.com/paper/2206.05323/) (2022)
- [AdaFlood: Adaptive Flood Regularization](https://mlsanity.com/paper/2311.02891/) (2023)
- [Contrastive Learning and Cycle Consistency-based Transductive Transfer Learning for Target Annotation](https://mlsanity.com/paper/2401.12340/) (2024)
- [The Paradox of Noise: An Empirical Study of Noise-Infusion Mechanisms to Improve Generalization, Stability, and Privacy in Federated Learning](https://mlsanity.com/paper/2311.05790/) (2023)
- [Curvature Enhanced Data Augmentation for Regression](https://mlsanity.com/paper/2506.06853/) (2025)
- [On gradient descent training under data augmentation with on-line noisy copies](https://mlsanity.com/paper/2206.03734/) (2022)
- [ALUM: Adversarial Data Uncertainty Modeling from Latent Model Uncertainty Compensation](https://mlsanity.com/paper/2303.16866/) (2023)
- [ShuffleMix: Improving Representations via Channel-Wise Shuffle of Interpolated Hidden States](https://mlsanity.com/paper/2305.18684/) (2023)
- [SNOiC: Soft Labeling and Noisy Mixup based Open Intent Classification Model](https://mlsanity.com/paper/2310.07306/) (2023)
- [Energy Guided smoothness to improve Robustness in Graph Classification](https://mlsanity.com/paper/2412.08419/) (2024)
- [Logit Mixture Outlier Exposure for Fine-grained Out-of-Distribution Detection](https://mlsanity.com/paper/2509.11892/) (2025)
- [Stylized Synthetic Augmentation further improves Corruption Robustness](https://mlsanity.com/paper/2512.15675/) (2025)
- [Isotropic Embedding Perturbations for Robust Vision Language Encoders](https://mlsanity.com/paper/2609.10292/) (2026)

## Abstract (arXiv)

We introduce Noisy Feature Mixup (NFM), an inexpensive yet effective method for data augmentation that combines the best of interpolation based training and noise injection schemes. Rather than training with convex combinations of pairs of examples and their labels, we use noise-perturbed convex combinations of pairs of data points in both input and feature space. This method includes mixup and manifold mixup as special cases, but it has additional advantages, including better smoothing of decision boundaries and enabling improved model robustness. We provide theory to understand this as well as the implicit regularization effects of NFM. Our theory is supported by empirical results, demonstrating the advantage of NFM, as compared to mixup and manifold mixup. We show that residual networks and vision transformers trained with NFM have favorable trade-offs between predictive accuracy on clean data and robustness with respect to various types of data perturbation across a range of computer vision benchmark datasets.