mlsanityData to 7 Oct 2026

Real World Robustness from Systematic Noise

arXiv 2109.00864 ↗First posted 2 Sep 2021cs.CVcited by 5

Not on the map. Why?

Abstract (arXiv)

Systematic error, which is not determined by chance, often refers to the inaccuracy (involving either the observation or measurement process) inherent to a system. In this paper, we exhibit some long-neglected but frequent-happening adversarial examples caused by systematic error. More specifically, we find the trained neural network classifier can be fooled by inconsistent implementations of image decoding and resize. This tiny difference between these implementations often causes an accuracy drop from training to deployment. To benchmark these real-world adversarial examples, we propose ImageNet-S dataset, which enables researchers to measure a classifier's robustness to systematic error. For example, we find a normal ResNet-50 trained on ImageNet can have 1%-5% accuracy difference due to the systematic error. Together our evaluation and dataset may aid future work toward real-world robustness and practical generalization.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 20 of its 29 references that are in the map. First posted on arXiv 2 Sep 2021.

Builds on

20 of 29 references.

References · 20

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  5. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  6. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  7. 2013Intriguing properties of neural networkscited by 5,114
  8. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  9. 2015Domain-Adversarial Training of Neural Networkscited by 3,023
  10. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  11. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  12. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
8 more
  1. 2019Natural Adversarial Examplescited by 1,401
  2. 2020Designing Network Design Spacescited by 954
  3. 2017Ensemble Adversarial Training: Attacks and Defensescited by 833
  4. 2019Adversarial Training for Free!cited by 638
  5. 2020ContextNet: Improving Convolutional Neural Networks for Automatic Speech Recognition with Global Contextcited by 96
  6. 2019Jasper: An End-to-End Convolutional Neural Acoustic Modelcited by 75
  7. 2019Are Adversarial Robustness and Common Perturbation Robustness Independent Attributes ?cited by 25
  8. 2020Label Smoothing and Adversarial Robustnesscited by 10

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 0; 2023: 3; 2024: 1.

5 papers.

  1. 2021RobustART: Benchmarking Robustness on Architecture Design and Training Techniquescited by 78
  2. 2024Can Biases in ImageNet Models Explain Generalization?cited by 20
  3. 2023RobustMQ: Benchmarking Robustness of Quantized Modelscited by 19
  4. 2023Benchmarking the Robustness of Quantized Modelscited by 2
  5. 2023SysNoise: Exploring and Benchmarking Training-Deployment System Inconsistencycited by 0