mlsanityData to 7 Oct 2026

Benchmarking Neural Network Robustness to Common Corruptions and Perturbations

arXiv 1903.12261 ↗First posted 28 Mar 2019cs.LG cs.CV stat.MLcited by 2,857

Not on the map. Why?

Cited by 130 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In this paper we establish rigorous benchmarks for image classifier robustness. Our first benchmark, ImageNet-C, standardizes and expands the corruption robustness topic, while showing which classifiers are preferable in safety-critical applications. Then we propose a new dataset called ImageNet-P which enables researchers to benchmark a classifier's robustness to common perturbations. Unlike recent robustness research, this benchmark evaluates performance on common corruptions and perturbations not worst-case adversarial perturbations. We find that there are negligible changes in relative corruption robustness from AlexNet classifiers to ResNet classifiers. Afterward we discover ways to enhance corruption and perturbation robustness. We even find that a bypassed adversarial defense provides substantial common perturbation robustness. Together our benchmarks may aid future work toward networks that robustly generalize.

Key facts. Cited by 2,857 papers in the map (data through 7 Oct 2026). First posted on arXiv 28 Mar 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 368; 2022: 480; 2023: 564; 2024: 541; 2025: 442; 2026: 461.

2,857 papers.

Sort:

By area:

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  3. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  4. 2022A ConvNet for the 2020scited by 3,545
  5. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  6. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,163
  7. 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 891
  8. 2021Robust fine-tuning of zero-shot modelscited by 785
  9. 2021Domain Generalization: A Surveycited by 760
  10. 2020Aligning AI With Shared Human Valuescited by 706
  11. 2023Scaling Vision Transformers to 22 Billion Parameterscited by 690
  12. 2021Extract Free Dense Labels from CLIPcited by 580
38 more
  1. 2021Domain Generalization with MixStylecited by 563
  2. 2022Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionalitycited by 523
  3. 2022Continual Test-Time Domain Adaptationcited by 520
  4. 2021Label-Efficient Semantic Segmentation with Diffusion Modelscited by 495
  5. 2021ReAct: Out-of-distribution Detection With Rectified Activationscited by 486
  6. 2022Efficient Test-Time Model Adaptation without Forgettingcited by 467
  7. 2021Intriguing Properties of Vision Transformerscited by 446
  8. 2021SWAD: Domain Generalization by Seeking Flat Minimacited by 431
  9. 2023Towards Stable Test-Time Adaptation in Dynamic Wild Worldcited by 409
  10. 2021Revisiting the Calibration of Modern Neural Networkscited by 387
  11. 2022Last Layer Re-Training is Sufficient for Robustness to Spurious Correlationscited by 379
  12. 2021MEMO: Test Time Robustness via Adaptation and Augmentationcited by 376
  13. 2021Laplace Redux -- Effortless Bayesian Deep Learningcited by 359
  14. 2021On the Importance of Gradients for Detecting Distributional Shifts in the Wildcited by 338
  15. 2021Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scanscited by 336
  16. 2021What Are Bayesian Neural Network Posteriors Really Like?cited by 307
  17. 2022How Do Vision Transformers Work?cited by 305
  18. 2022Masked Siamese Networks for Label-Efficient Learningcited by 301
  19. 2021DAFormer: Improving Network Architectures and Training Strategies for Domain-Adaptive Semantic Segmentationcited by 299
  20. 2022Scaling & Shifting Your Features: A New Baseline for Efficient Model Tuningcited by 292
  21. 2021Unsolved Problems in ML Safetycited by 291
  22. 2022MultiMAE: Multi-modal Multi-task Masked Autoencoderscited by 287
  23. 2023Blind Image Quality Assessment via Vision-Language Correspondence: A Multitask Learning Perspectivecited by 276
  24. 2022Cold Diffusion: Inverting Arbitrary Image Transforms Without Noisecited by 275
  25. 2021When Vision Transformers Outperform ResNets without Pre-training or Strong Data Augmentationscited by 259
  26. 2021Global Filter Networks for Image Classificationcited by 257
  27. 2021Understanding Robustness of Transformers for Image Classificationcited by 250
  28. 2021Accuracy on the Line: On the Strong Correlation Between Out-of-Distribution and In-Distribution Generalizationcited by 248
  29. 2023AdaMerging: Adaptive Model Merging for Multi-Task Learningcited by 248
  30. 2024Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications and Opportunitiescited by 247
  31. 2021Improving Robustness using Generated Datacited by 233
  32. 2022DeepFusion: Lidar-Camera Deep Fusion for Multi-Modal 3D Object Detectioncited by 224
  33. 2021Vision Transformers are Robust Learnerscited by 208
  34. 2022Test-Time Training with Masked Autoencoderscited by 208
  35. 2021Towards Causal Representation Learningcited by 207
  36. 2023Better Diffusion Models Further Improve Adversarial Trainingcited by 207
  37. 2021Trustworthy AI: From Principles to Practicescited by 206
  38. 2022Surgical Fine-Tuning Improves Adaptation to Distribution Shiftscited by 205

50 most cited of 2,857 All 2,857 (JSON)