# An Empirical Evaluation on Robustness and Uncertainty of Regularization Methods

arXiv: https://arxiv.org/abs/2003.03879 · HTML: https://mlsanity.com/paper/2003.03879/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Mar 2020.

## Built on by (most cited first, 28 of 28)

- [PixMix: Dreamlike Pictures Comprehensively Improve Safety Measures](https://mlsanity.com/paper/2112.05135/) (2021)
- [Mixture Outlier Exposure: Towards Out-of-Distribution Detection in Fine-grained Environments](https://mlsanity.com/paper/2106.03917/) (2021)
- [Learning Fair Classifiers with Partially Annotated Group Labels](https://mlsanity.com/paper/2111.14581/) (2021)
- [A Unified Analysis of Mixed Sample Data Augmentation: A Loss Function Perspective](https://mlsanity.com/paper/2208.09913/) (2022)
- [On the Effect of Pruning on Adversarial Robustness](https://mlsanity.com/paper/2108.04890/) (2021)
- [IPMix: Label-Preserving Data Augmentation Method for Training Robust Classifiers](https://mlsanity.com/paper/2310.04780/) (2023)
- [Towards Understanding Why Label Smoothing Degrades Selective Classification and How to Fix It](https://mlsanity.com/paper/2403.14715/) (2024)
- [A.I. Robustness: a Human-Centered Perspective on Technological Challenges and Opportunities](https://mlsanity.com/paper/2210.08906/) (2022)
- [SeiT: Storage-Efficient Vision Training with Tokens Using 1% of Pixel Storage](https://mlsanity.com/paper/2303.11114/) (2023)
- [Few-shot Font Generation with Weakly Supervised Localized Representations](https://mlsanity.com/paper/2112.11895/) (2021)
- [Observations on K-image Expansion of Image-Mixing Augmentation for Classification](https://mlsanity.com/paper/2110.04248/) (2021)
- [Similarity of Neural Architectures using Adversarial Attack Transferability](https://mlsanity.com/paper/2210.11407/) (2022)
- [Navigating Noise: A Study of How Noise Influences Generalisation and Calibration of Neural Networks](https://mlsanity.com/paper/2306.17630/) (2023)
- [Revealing the Two Sides of Data Augmentation: An Asymmetric Distillation-based Win-Win Solution for Open-Set Recognition](https://mlsanity.com/paper/2404.19527/) (2024)
- [Diverse Gaussian Noise Consistency Regularization for Robustness and Uncertainty Calibration](https://mlsanity.com/paper/2104.01231/) (2021)
- [StyleAugment: Learning Texture De-biased Representations by Style Augmentation without Pre-defined Textures](https://mlsanity.com/paper/2108.10549/) (2021)
- [On the Effects of Artificial Data Modification](https://mlsanity.com/paper/2110.13968/) (2021)
- [Analyzing Effects of Mixed Sample Data Augmentation on Model Interpretability](https://mlsanity.com/paper/2303.14608/) (2023)
- [Frustratingly Easy Uncertainty Estimation for Distribution Shift](https://mlsanity.com/paper/2106.03762/) (2021)
- [Learning from Matured Dumb Teacher for Fine Generalization](https://mlsanity.com/paper/2108.05776/) (2021)
- [SeiT++: Masked Token Modeling Improves Storage-efficient Training](https://mlsanity.com/paper/2312.10105/) (2023)
- [Research Trends and Applications of Data Augmentation Algorithms](https://mlsanity.com/paper/2207.08817/) (2022)
- [Exploring the Design of Adaptation Protocols for Improved Generalization and Machine Learning Safety](https://mlsanity.com/paper/2207.12615/) (2022)
- [On Pitfalls of Measuring Occlusion Robustness through Data Distortion](https://mlsanity.com/paper/2211.13734/) (2022)
- [MixBoost: Improving the Robustness of Deep Neural Networks by Boosting Data Augmentation](https://mlsanity.com/paper/2212.04059/) (2022)
- [Enhancing Multiple Reliability Measures via Nuisance-extended Information Bottleneck](https://mlsanity.com/paper/2303.14096/) (2023)
- [On the Equivalence of Graph Convolution and Mixup](https://mlsanity.com/paper/2310.00183/) (2023)
- [Hallucination Detection in Virtually-Stained Histology: A Latent Space Baseline](https://mlsanity.com/paper/2411.15060/) (2024)

## Abstract (arXiv)

Despite apparent human-level performances of deep neural networks (DNN), they behave fundamentally differently from humans. They easily change predictions when small corruptions such as blur and noise are applied on the input (lack of robustness), and they often produce confident predictions on out-of-distribution samples (improper uncertainty measure). While a number of researches have aimed to address those issues, proposed solutions are typically expensive and complicated (e.g. Bayesian inference and adversarial training). Meanwhile, many simple and cheap regularization methods have been developed to enhance the generalization of classifiers. Such regularization methods have largely been overlooked as baselines for addressing the robustness and uncertainty issues, as they are not specifically designed for that. In this paper, we provide extensive empirical evaluations on the robustness and uncertainty estimates of image classifiers (CIFAR-100 and ImageNet) trained with state-of-the-art regularization methods. Furthermore, experimental results show that certain regularization methods can serve as strong baseline methods for robustness and uncertainty estimation of DNNs.