mlsanityData to 14 Oct 2026

RegMixup: Mixup as a Regularizer Can Surprisingly Improve Accuracy and Out Distribution Robustness

arXiv 2206.14502 ↗First posted 29 Jun 2022cs.LG cs.CVcited by 27

Not on the map. Why?

Abstract (arXiv)

We show that the effectiveness of the well celebrated Mixup [Zhang et al., 2018] can be further improved if instead of using it as the sole learning objective, it is utilized as an additional regularizer to the standard cross-entropy loss. This simple change not only provides much improved accuracy but also significantly improves the quality of the predictive uncertainty estimation of Mixup in most cases under various forms of covariate shifts and out-of-distribution detection experiments. In fact, we observe that Mixup yields much degraded performance on detecting out-of-distribution samples possibly, as we show empirically, because of its tendency to learn models that exhibit high-entropy throughout; making it difficult to differentiate in-distribution samples from out-distribution ones. To show the efficacy of our approach (RegMixup), we provide thorough analyses and experiments on vision datasets (ImageNet & CIFAR-10/100) and compare it with a suite of recent approaches for reliable uncertainty estimation.

Key facts. Cited by 27 papers in the map (data through 14 Oct 2026). Builds on 48 of its 70 references that are in the map. First posted on arXiv 29 Jun 2022.

Builds on

48 of 70 references.

References · 48

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2016Densely Connected Convolutional Networkscited by 6,720
  4. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  5. 2017On Calibration of Modern Neural Networkscited by 4,960
  6. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  7. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  8. 2016Wide Residual Networkscited by 3,070
  9. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,884
  10. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
  11. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,850
  12. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,636
36 more
  1. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,515
  2. 2019Natural Adversarial Examplescited by 1,408
  3. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,340
  4. 2020Energy-based Out-of-distribution Detectioncited by 1,299
  5. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,287
  6. 2019Learning Robust Global Representations by Penalizing Local Predictive Powercited by 1,013
  7. 2019When Does Label Smoothing Help?cited by 988
  8. 2019AugMix: A Simple Data Processing Method to Improve Robustness and Uncertaintycited by 901
  9. 2021Generalizing to Unseen Domains: A Survey on Domain Generalizationcited by 766
  10. 2018Evidential Deep Learning to Quantify Classification Uncertaintycited by 732
  11. 2019Using Self-Supervised Learning Can Improve Model Robustness and Uncertaintycited by 514
  12. 2015Variational Dropout and the Local Reparameterization Trickcited by 490
  13. 2018Certified Robustness to Adversarial Examples with Differential Privacycited by 468
  14. 2020Measuring Robustness to Natural Distribution Shifts in Image Classificationcited by 454
  15. 2020Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awarenesscited by 389
  16. 2020BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learningcited by 371
  17. 2020Calibrating Deep Neural Networks using Focal Losscited by 357
  18. 2017Snapshot Ensembles: Train 1, get M for freecited by 350
  19. 2019SelectiveNet: A Deep Neural Network with an Integrated Reject Optioncited by 350
  20. 2014Stochastic Gradient Hamiltonian Monte Carlocited by 343
  21. 2019On Mixup Training: Improved Calibration and Predictive Uncertainty for Deep Neural Networkscited by 305
  22. 2021Gradient Matching for Domain Generalizationcited by 258
  23. 2020Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networkscited by 231
  24. 2020Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learningcited by 210
  25. 2019Cyclical Stochastic Gradient MCMC for Bayesian Deep Learningcited by 203
  26. 2022Improving Out-of-Distribution Robustness via Selective Augmentationcited by 202
  27. 2018Do CIFAR-10 Classifiers Generalize to CIFAR-10?cited by 200
  28. 2020Training independent subnetworks for robust predictioncited by 177
  29. 2016Deep Pyramidal Residual Networkscited by 163
  30. 2020Uncertainty Quantification and Deep Ensemblescited by 139
  31. 2019BackPACK: Packing more into backpropcited by 78
  32. 2020Beyond Pinball Loss: Quantile Methods for Calibrated Uncertainty Quantificationcited by 78
  33. 2022Connect, Not Collapse: Explaining Contrastive Learning for Unsupervised Domain Adaptationcited by 67
  34. 2020Combining Ensembles and Data Augmentation can Harm your Calibrationcited by 44
  35. 2019Stable Rank Normalization for Improved Generalization in Neural Networks and GANscited by 41
  36. 2020Fast Predictive Uncertainty for Classification with Bayesian Deep Networkscited by 31

Built on by

Cited by year

Citing papers per year: 2022: 6; 2023: 14; 2024: 5; 2025: 2.

27 papers.

Sort:

By area:

  1. 2022A Whac-A-Mole Dilemma: Shortcuts Come in Multiples Where Mitigating One Amplifies Otherscited by 85
  2. 2023Discover and Cure: Concept-aware Mitigation of Spurious Correlationcited by 71
  3. 2022Sample-dependent Adaptive Temperature Scaling for Improved Calibrationcited by 48
  4. 2023OpenMix: Exploring Outlier Samples for Misclassification Detectioncited by 38
  5. 2022A Survey of Mix-based Data Augmentation: Taxonomy, Methods, Applications, and Explainabilitycited by 31
  6. 2023Towards Building Self-Aware Object Detectors via Reliable Uncertainty Quantification and Calibrationcited by 19
  7. 2023A Data-Driven Measure of Relative Uncertainty for Misclassification Detectioncited by 14
  8. 2023Over-training with Mixup May Hurt Generalizationcited by 13
  9. 2023Does Progress On Object Recognition Benchmarks Improve Real-World Generalization?cited by 13
  10. 2024Multimodality Invariant Learning for Multimedia-Based New Item Recommendationcited by 13
  11. 2023Selective Mixup Helps with Distribution Shifts, But Not (Only) because of Mixupcited by 8
  12. 2022Raising the Bar on the Evaluation of Out-of-Distribution Detectioncited by 7
15 more
  1. 2024OpenCIL: Benchmarking Out-of-Distribution Detection in Class-Incremental Learningcited by 7
  2. 2023Tailoring Mixup to Data for Calibrationcited by 5
  3. 2023Class Uncertainty: A Measure to Mitigate Class Imbalancecited by 4
  4. 2024Improving EEG Classification Through Randomly Reassembling Original and Generated Data with Transformer-based Diffusion Modelscited by 4
  5. 2023Supervision Interpolation via LossMix: Generalizing Mixup for Object Detection and Beyondcited by 3
  6. 2022Query-based Hard-Image Retrieval for Object Detection at Test Timecited by 2
  7. 2023Infinite Class Mixupcited by 2
  8. 2023A Metacognitive Approach to Out-of-Distribution Detection for Segmentationcited by 2
  9. 2025OpenEarthSensing: Large-Scale Fine-Grained Benchmark for Open-World Remote Sensingcited by 2
  10. 2023A Comprehensive Assessment Benchmark for Rigorously Evaluating Deep Learning Image Classifierscited by 1
  11. 2022SimpleDG: Simple Domain Generalization Baseline without Bells and Whistlescited by 0
  12. 2023Enhancing Visual Domain Adaptation with Source Preparationcited by 0
  13. 2024TriAug: Out-of-Distribution Detection for Imbalanced Breast Lesion in Ultrasoundcited by 0
  14. 2024Mix from Failure: Confusion-Pairing Mixup for Long-Tailed Recognitioncited by 0
  15. 2025TrustLoRA: Low-Rank Adaptation for Failure Detection under Out-of-distribution Datacited by 0