# Now You See It, Now You Dont: Adversarial Vulnerabilities in Computational Pathology

arXiv: https://arxiv.org/abs/2106.08153 · HTML: https://mlsanity.com/paper/2106.08153/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 35 references that are in the map.
- First posted on arXiv 14 Jun 2021.

## Builds on (most cited first, 22 of 22)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Adversarial Examples Are Not Bugs, They Are Features](https://mlsanity.com/paper/1905.02175/) (2019)
- [The Limitations of Deep Learning in Adversarial Settings](https://mlsanity.com/paper/1511.07528/) (2015)
- [Universal adversarial perturbations](https://mlsanity.com/paper/1610.08401/) (2016)
- [Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey](https://mlsanity.com/paper/1801.00553/) (2018)
- [On Evaluating Adversarial Robustness](https://mlsanity.com/paper/1902.06705/) (2019)
- [HoVer-Net: Simultaneous Segmentation and Classification of Nuclei in Multi-Tissue Histology Images](https://mlsanity.com/paper/1812.06499/) (2018)
- [Confounding variables can degrade generalization performance of radiological deep learning models](https://mlsanity.com/paper/1807.00431/) (2018)
- [High Frequency Component Helps Explain the Generalization of Convolutional Neural Networks](https://mlsanity.com/paper/1905.13545/) (2019)
- [What makes ImageNet good for transfer learning?](https://mlsanity.com/paper/1608.08614/) (2016)
- [Not-so-supervised: a survey of semi-supervised, multi-instance, and transfer learning in medical image analysis](https://mlsanity.com/paper/1804.06353/) (2018)
- [Understanding Adversarial Attacks on Deep Learning Based Medical Image Analysis Systems](https://mlsanity.com/paper/1907.10456/) (2019)
- [PanNuke Dataset Extension, Insights and Baselines](https://mlsanity.com/paper/2003.10778/) (2020)
- [On the Connection Between Adversarial Robustness and Saliency Map Interpretability](https://mlsanity.com/paper/1905.04172/) (2019)
- [Adversarial Attacks Against Medical Deep Learning Systems](https://mlsanity.com/paper/1804.05296/) (2018)
- [Detecting Overfitting via Adversarial Examples](https://mlsanity.com/paper/1903.02380/) (2019)

## Built on by (most cited first, 7 of 7)

- [CoNIC: Colon Nuclei Identification and Counting Challenge 2022](https://mlsanity.com/paper/2111.14485/) (2021)
- [Survey on Adversarial Attack and Defense for Medical Image Analysis: Methods and Challenges](https://mlsanity.com/paper/2303.14133/) (2023)
- [REET: Robustness Evaluation and Enhancement Toolbox for Computational Pathology](https://mlsanity.com/paper/2201.12311/) (2022)
- [Artifact-Robust Graph-Based Learning in Digital Pathology](https://mlsanity.com/paper/2310.18192/) (2023)
- [Robust and Interpretable COVID-19 Diagnosis on Chest X-ray Images using Adversarial Training](https://mlsanity.com/paper/2311.14227/) (2023)
- [Adversary-Robust Graph-Based Learning of WSIs](https://mlsanity.com/paper/2403.14489/) (2024)
- [Hierarchical Self-Supervised Adversarial Training for Robust Vision Models in Histopathology](https://mlsanity.com/paper/2503.10629/) (2025)

## Abstract (arXiv)

Deep learning models are routinely employed in computational pathology (CPath) for solving problems of diagnostic and prognostic significance. Typically, the generalization performance of CPath models is analyzed using evaluation protocols such as cross-validation and testing on multi-centric cohorts. However, to ensure that such CPath solutions are robust and safe for use in a clinical setting, a critical analysis of their predictive performance and vulnerability to adversarial attacks is required, which is the focus of this paper. Specifically, we show that a highly accurate model for classification of tumour patches in pathology images (AUC > 0.95) can easily be attacked with minimal perturbations which are imperceptible to lay humans and trained pathologists alike. Our analytical results show that it is possible to generate single-instance white-box attacks on specific input images with high success rate and low perturbation energy. Furthermore, we have also generated a single universal perturbation matrix using the training dataset only which, when added to unseen test images, results in forcing the trained neural network to flip its prediction labels with high confidence at a success rate of > 84%. We systematically analyze the relationship between perturbation energy of an adversarial attack, its impact on morphological constructs of clinical significance, their perceptibility by a trained pathologist and saliency maps obtained using deep learning models. Based on our analysis, we strongly recommend that computational pathology models be critically analyzed using the proposed adversarial validation strategy prior to clinical adoption.