mlsanityData to 7 Oct 2026

On Evaluating Adversarial Robustness of Chest X-ray Classification: Pitfalls and Best Practices

arXiv 2212.08130 ↗First posted 15 Dec 2022eess.IV cs.CV cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

Vulnerability to adversarial attacks is a well-known weakness of Deep Neural Networks. While most of the studies focus on natural images with standardized benchmarks like ImageNet and CIFAR, little research has considered real world applications, in particular in the medical domain. Our research shows that, contrary to previous claims, robustness of chest x-ray classification is much harder to evaluate and leads to very different assessments based on the dataset, the architecture and robustness metric. We argue that previous studies did not take into account the peculiarity of medical diagnosis, like the co-occurrence of diseases, the disagreement of labellers (domain experts), the threat model of the attacks and the risk implications for each successful attack. In this paper, we discuss the methodological foundations, review the pitfalls and best practices, and suggest new methodological considerations for evaluating the robustness of chest xray classification models. Our evaluation on 3 datasets, 7 models, and 18 diseases is the largest evaluation of robustness of chest x-ray classification models.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 29 of its 54 references that are in the map. First posted on arXiv 15 Dec 2022.

Builds on

29 of 54 references.

References · 29

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  3. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
  4. 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,410
  5. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
  6. 2017ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseasescited by 969
  7. 2016Adversarial Machine Learning at Scalecited by 934
  8. 2017Evasion Attacks against Machine Learning at Test Timecited by 686
  9. 2020RobustBench: a standardized adversarial robustness benchmarkcited by 659
  10. 2017CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learningcited by 459
  11. 2019On Evaluating Adversarial Robustnesscited by 442
  12. 2019PadChest: A large chest x-ray image dataset with multi-label annotated reportscited by 308
17 more
  1. 2019Understanding Adversarial Attacks on Deep Learning Based Medical Image Analysis Systemscited by 151
  2. 2018Adversarial Attacks Against Medical Deep Learning Systemscited by 61
  3. 2020On the limits of cross-domain generalization in automated X-ray predictioncited by 61
  4. 2018Comparison of Deep Learning Approaches for Multi-Label Chest X-Ray Classificationcited by 57
  5. 2019Can we trust deep learning models diagnosis? The impact of domain shift in chest radiograph classificationcited by 57
  6. 2019Exploring large scale public medical image datasetscited by 37
  7. 2018Generalizability vs. Robustness: Adversarial Examples for Medical Imagingcited by 23
  8. 2021A Unified Framework for Adversarial Attack and Defense in Constrained Feature Spacecited by 22
  9. 2019Multi-Label Adversarial Perturbationscited by 18
  10. 2021Adversarial Robustness in Multi-Task Learning: Promises and Illusionscited by 17
  11. 2021Towards Evaluating the Robustness of Deep Diagnostic Models by Adversarial Attackcited by 15
  12. 2019A Strong Baseline for Domain Adaptation and Generalization in Medical Imagingcited by 11
  13. 2020Bias Field Poses a Threat to DNN-based X-Ray Recognitioncited by 11
  14. 2018Vulnerability Analysis of Chest X-Ray Image Classification Against Adversarial Attackscited by 9
  15. 2019Influence of Control Parameters and the Size of Biomedical Image Datasets on the Success of Adversarial Attackscited by 2
  16. 2020Vulnerability of deep neural networks for detecting COVID-19 cases from chest X-ray images to universal adversarial attackscited by 2
  17. 2020Fuzzy Unique Image Transformation: Defense Against Adversarial Attacks On Deep COVID-19 Modelscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 1.

3 papers.

  1. 2024Tilt your Head: Activating the Hidden Spatial-Invariance of Classifierscited by 7
  2. 2025Brain Imaging Foundation Models, Are We There Yet? A Systematic Review of Foundation Models for Brain Imaging and Biomedical Researchcited by 0
  3. 2026Text-Guided Diffusion-Based Adversarial Attacks on Chest X-Ray Imagescited by 0