mlsanityData to 7 Oct 2026

Benchmarking Neural Network Robustness to Common Corruptions and Surface Variations

arXiv 1807.01697 ↗First posted 4 Jul 2018cs.LG cs.AI cs.CVcited by 99

Not on the map. Why?

Abstract (arXiv)

In this paper we establish rigorous benchmarks for image classifier robustness. Our first benchmark, ImageNet-C, standardizes and expands the corruption robustness topic, while showing which classifiers are preferable in safety-critical applications. Unlike recent robustness research, this benchmark evaluates performance on commonplace corruptions not worst-case adversarial corruptions. We find that there are negligible changes in relative corruption robustness from AlexNet to ResNet classifiers, and we discover ways to enhance corruption robustness. Then we propose a new dataset called Icons-50 which opens research on a new kind of robustness, surface variation robustness. With this dataset we evaluate the frailty of classifiers on new styles of known objects and unexpected instances of known classes. We also demonstrate two methods that improve surface variation robustness. Together our benchmarks may aid future work toward networks that learn fundamental class structure and also robustly generalize.

Key facts. Cited by 99 papers in the map (data through 7 Oct 2026). First posted on arXiv 4 Jul 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 14; 2022: 22; 2023: 23; 2024: 20; 2025: 17; 2026: 3.

99 papers.

Sort:

By area:

  1. 2021Are Transformers More Robust Than CNNs?cited by 187
  2. 2025DriveSuprim: Towards Precise Trajectory Selection for End-to-End Planningcited by 91
  3. 2022Distilling Model Failures as Directions in Latent Spacecited by 82
  4. 2022Reliability of CKA as a Similarity Measure in Deep Learningcited by 82
  5. 2021Editing a classifier by rewriting its prediction rulescited by 64
  6. 2022Deep Ensembles Work, But Are They Necessary?cited by 63
  7. 2023FACET: Fairness in Computer Vision Evaluation Benchmarkcited by 58
  8. 2023Feed Two Birds with One Scone: Exploiting Wild Data for Both Out-of-Distribution Generalization and Detectioncited by 57
  9. 2021Negative Data Augmentationcited by 46
  10. 2022Generalized Parametric Contrastive Learningcited by 45
  11. 2023In Search of Lost Online Test-time Adaptation: A Surveycited by 43
  12. 2021SECANT: Self-Expert Cloning for Zero-Shot Generalization of Visual Policiescited by 42
38 more
  1. 2022Test-Time Robust Personalization for Federated Learningcited by 39
  2. 2022Can CNNs Be More Robust Than Transformers?cited by 38
  3. 2022Missingness Bias in Model Debuggingcited by 36
  4. 2023Dataset Interfaces: Diagnosing Model Failures Using Controllable Counterfactual Generationcited by 35
  5. 2023Dual Focal Loss for Calibrationcited by 33
  6. 20213DB: A Framework for Debugging Computer Vision Modelscited by 31
  7. 2024COME: Test-time adaption by Conservatively Minimizing Entropycited by 31
  8. 2024B-AVIBench: Towards Evaluating the Robustness of Large Vision-Language Model on Black-box Adversarial Visual-Instructionscited by 26
  9. 2022Are Deep Neural Networks SMARTer than Second Graders?cited by 24
  10. 2024Decomposing and Editing Predictions by Modeling Model Computationcited by 19
  11. 2023FLAC: Fairness-Aware Representation Learning by Suppressing Attribute-Class Associationscited by 18
  12. 2023Pathologies of Predictive Diversity in Deep Ensemblescited by 16
  13. 2023PerceptionCLIP: Visual Classification by Inferring and Conditioning on Contextscited by 16
  14. 2023The Five-Dollar Model: Generating Game Maps and Sprites from Sentence Embeddingscited by 14
  15. 2024FOOGD: Federated Collaboration for Both Out-of-distribution Generalization and Detectioncited by 14
  16. 2025Out-of-Distribution Detection Methods Answer the Wrong Questionscited by 14
  17. 2025Componential Prompt-Knowledge Alignment for Domain Incremental Learningcited by 13
  18. 2024Multilinear Operator Networkscited by 12
  19. 2021LiftPool: Bidirectional ConvNet Poolingcited by 11
  20. 2022Metaethical Perspectives on 'Benchmarking' AI Ethicscited by 11
  21. 2023FAIR-Ensemble: When Fairness Naturally Emerges From Deep Ensemblingcited by 11
  22. 2023Unraveling Batch Normalization for Realistic Test-Time Adaptationcited by 10
  23. 2025Vision-Language Model for Object Detection and Segmentation: A Review and Evaluationcited by 10
  24. 2023Calibrating a Deep Neural Network with Its Predecessorscited by 9
  25. 2024Posterior Uncertainty Quantification in Neural Networks using Data Augmentationcited by 9
  26. 2021Domain Invariant Adversarial Learningcited by 7
  27. 2021SI-Score: An image dataset for fine-grained analysis of robustness to object location, rotation and sizecited by 7
  28. 2024Dynamics-Guided Diffusion Model for Sensor-less Robot Manipulator Designcited by 7
  29. 2024Distance-Forward Learning: Enhancing the Forward-Forward Algorithm Towards High-Performance On-Chip Learningcited by 7
  30. 2024AHA: Human-Assisted Out-of-Distribution Generalization and Detectioncited by 7
  31. 2022Fast AdvPropcited by 6
  32. 2024Out-of-Distribution Learning with Human Feedbackcited by 6
  33. 2021Road Scenes Segmentation Across Different Domains by Disentangling Latent Representationscited by 5
  34. 2022What does a deep neural network confidently perceive? The effective dimension of high certainty class manifolds and their low confidence boundariescited by 5
  35. 2023Asymmetric Bias in Text-to-Image Generation with Adversarial Attackscited by 5
  36. 2024Investigating Calibration and Corruption Robustness of Post-hoc Pruned Perception CNNs: An Image Classification Benchmark Studycited by 5
  37. 2024BAdd: Bias Mitigation through Bias Additioncited by 5
  38. 2022On Certifying and Improving Generalization to Unseen Domainscited by 4

50 most cited of 99 All 99 (JSON)