mlsanityData to 13 Oct 2026

Noisy Feature Mixup

arXiv 2110.02180 ↗First posted 5 Oct 2021cs.LG stat.MLcited by 27

Not on the map. Why?

Abstract (arXiv)

We introduce Noisy Feature Mixup (NFM), an inexpensive yet effective method for data augmentation that combines the best of interpolation based training and noise injection schemes. Rather than training with convex combinations of pairs of examples and their labels, we use noise-perturbed convex combinations of pairs of data points in both input and feature space. This method includes mixup and manifold mixup as special cases, but it has additional advantages, including better smoothing of decision boundaries and enabling improved model robustness. We provide theory to understand this as well as the implicit regularization effects of NFM. Our theory is supported by empirical results, demonstrating the advantage of NFM, as compared to mixup and manifold mixup. We show that residual networks and vision transformers trained with NFM have favorable trade-offs between predictive accuracy on clean data and robustness with respect to various types of data perturbation across a range of computer vision benchmark datasets.

Key facts. Cited by 27 papers in the map (data through 13 Oct 2026). Builds on 51 of its 82 references that are in the map. First posted on arXiv 5 Oct 2021.

Builds on

51 of 82 references.

References · 51

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,336
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,379
  3. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,170
  4. 2016Wide Residual Networkscited by 3,063
  5. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,882
  6. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,862
  7. 2016Identity Mappings in Deep Residual Networkscited by 1,962
  8. 2016Adversarial examples in the physical worldcited by 1,767
  9. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,511
  10. 2019AugMix: A Simple Data Processing Method to Improve Robustness and Uncertaintycited by 899
  11. 2018Robustness May Be at Odds with Accuracycited by 878
  12. 2020Overfitting in adversarially robust deep learningcited by 505
38 more
  1. 2018Certified Robustness to Adversarial Examples with Differential Privacycited by 467
  2. 2018Adversarially Robust Generalization Requires More Datacited by 318
  3. 2019On Mixup Training: Improved Calibration and Predictive Uncertainty for Deep Neural Networkscited by 304
  4. 2021Escaping the Big Data Paradigm with Compact Transformerscited by 263
  5. 2020Puzzle Mix: Exploiting Saliency and Local Statistics for Optimal Mixupcited by 261
  6. 2021Vision Transformers are Robust Learnerscited by 207
  7. 2021Fixing Data Augmentation to Improve Adversarial Robustnesscited by 203
  8. 2020How Does Mixup Help With Robustness and Generalization?cited by 193
  9. 2018Sensitivity and Generalization in Neural Networks: an Empirical Studycited by 187
  10. 2017Formal Guarantees on the Robustness of a Classifier against Adversarial Manipulationcited by 182
  11. 2018Robustness via curvature regularization, and vice versacited by 174
  12. 2019Wasserstein Distributionally Robust Optimization: Theory and Applications in Machine Learningcited by 170
  13. 2020Understanding and Mitigating the Tradeoff Between Robustness and Accuracycited by 149
  14. 2018Is Robustness the Cost of Accuracy? -- A Comprehensive Study on the Robustness of 18 Deep Image Classification Modelscited by 135
  15. 2019Robust Learning with Jacobian Regularizationcited by 127
  16. 2016Robust Large Margin Deep Neural Networkscited by 117
  17. 2018Large Margin Deep Networks for Classificationcited by 112
  18. 2018Predicting the Generalization Gap in Deep Networks with Margin Distributionscited by 112
  19. 2018A Kernel Theory of Modern Data Augmentationcited by 106
  20. 2016Robustness of classifiers: from adversarial to random noisecited by 87
  21. 2020On Mixup Regularizationcited by 82
  22. 2020In Search of Robust Measures of Generalizationcited by 79
  23. 2019Data-dependent Sample Complexity of Deep Neural Networks via Lipschitz Augmentationcited by 71
  24. 2021When and How Mixup Improves Calibrationcited by 60
  25. 2019Improved Sample Complexities for Deep Networks and Robust Classification via an All-Layer Margincited by 56
  26. 2016Noisy Activation Functionscited by 53
  27. 2019Interpolated Adversarial Training: Achieving Robust Neural Networks without Sacrificing Too Much Accuracycited by 49
  28. 2020On the Generalization Effects of Linear Transformations in Data Augmentationcited by 47
  29. 2021Noisy Recurrent Neural Networkscited by 42
  30. 2020The Curious Case of Adversarially Robust Models: More Data Can Help, Double Descend, or Hurt Generalizationcited by 39
  31. 2020Explicit Regularisation in Gaussian Noise Injectionscited by 37
  32. 2019Theoretical evidence for adversarial robustness through randomizationcited by 33
  33. 2020MaxUp: A Simple Way to Improve Generalization of Neural Network Trainingcited by 27
  34. 2020Boundary thickness and robustness in learning modelscited by 26
  35. 2020Adversarially-Trained Deep Nets Transfer Better: Illustration on Image Classificationcited by 18
  36. 2020Principled learning method for Wasserstein distributionally robust optimization with local perturbationscited by 14
  37. 2021k-Mixup Regularization for Deep Learning via Optimal Transportcited by 14
  38. 2019A unified view on differential privacy and robustness to adversarial examplescited by 9

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2022: 5; 2023: 10; 2024: 6; 2025: 5; 2026: 1.

27 papers.

Sort:

By area:

  1. 2022A Survey of Mix-based Data Augmentation: Taxonomy, Methods, Applications, and Explainabilitycited by 31
  2. 2023FedFA: Federated Feature Augmentationcited by 22
  3. 2023DART: Diversify-Aggregate-Repeat Training Improves Generalization of Neural Networkscited by 20
  4. 2022NoisyMix: Boosting Model Robustness to Common Corruptionscited by 17
  5. 2025Concept Pinpoint Eraser for Text-to-image Diffusion Models via Residual Attention Gatecited by 16
  6. 2024First-Order Manifold Data Augmentation for Regression Learningcited by 6
  7. 2022Gated Recurrent Neural Networks with Weighted Time-Delay Feedbackcited by 5
  8. 2023Tailoring Mixup to Data for Calibrationcited by 5
  9. 2025Robust Asymmetric Heterogeneous Federated Learning with Corrupted Clientscited by 5
  10. 2023Investigating the Corruption Robustness of Image Classifiers with Random Lp-norm Corruptionscited by 4
  11. 2024Learning to Detour: Shortcut Mitigating Augmentation for Weakly Supervised Semantic Segmentationcited by 4
  12. 2023PDE+: Enhancing Generalization via PDE with Adaptive Distributional Diffusioncited by 3
15 more
  1. 2024For Better or For Worse? Learning Minimum Variance Features With Label Augmentationcited by 3
  2. 2024A Unified Deep Semantic Expansion Framework for Domain-Generalized Person Re-identificationcited by 3
  3. 2022Memory Classifiers: Two-stage Classification for Robustness in Machine Learningcited by 2
  4. 2023AdaFlood: Adaptive Flood Regularizationcited by 2
  5. 2024Contrastive Learning and Cycle Consistency-based Transductive Transfer Learning for Target Annotationcited by 2
  6. 2023The Paradox of Noise: An Empirical Study of Noise-Infusion Mechanisms to Improve Generalization, Stability, and Privacy in Federated Learningcited by 1
  7. 2025Curvature Enhanced Data Augmentation for Regressioncited by 1
  8. 2022On gradient descent training under data augmentation with on-line noisy copiescited by 0
  9. 2023ALUM: Adversarial Data Uncertainty Modeling from Latent Model Uncertainty Compensationcited by 0
  10. 2023ShuffleMix: Improving Representations via Channel-Wise Shuffle of Interpolated Hidden Statescited by 0
  11. 2023SNOiC: Soft Labeling and Noisy Mixup based Open Intent Classification Modelcited by 0
  12. 2024Energy Guided smoothness to improve Robustness in Graph Classificationcited by 0
  13. 2025Logit Mixture Outlier Exposure for Fine-grained Out-of-Distribution Detectioncited by 0
  14. 2025Stylized Synthetic Augmentation further improves Corruption Robustnesscited by 0
  15. 2026Isotropic Embedding Perturbations for Robust Vision Language Encoderscited by 0