# S-E Pipeline: A Vision Transformer (ViT) based Resilient Classification Pipeline for Medical Imaging Against Adversarial Attacks

arXiv: https://arxiv.org/abs/2407.17587 · HTML: https://mlsanity.com/paper/2407.17587/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 27 references that are in the map.
- First posted on arXiv 23 Jul 2024.

## Builds on (most cited first, 19 of 19)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Countering Adversarial Images using Input Transformations](https://mlsanity.com/paper/1711.00117/) (2017)
- [Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks](https://mlsanity.com/paper/1704.01155/) (2017)
- [A study of the effect of JPG compression on adversarial images](https://mlsanity.com/paper/1608.00853/) (2016)
- [On the Robustness of Vision Transformers to Adversarial Examples](https://mlsanity.com/paper/2104.02610/) (2021)
- [Understanding Adversarial Attacks on Deep Learning Based Medical Image Analysis Systems](https://mlsanity.com/paper/1907.10456/) (2019)
- [LViT: Language meets Vision Transformer in Medical Image Segmentation](https://mlsanity.com/paper/2206.14718/) (2022)
- [On the Adversarial Robustness of Vision Transformers](https://mlsanity.com/paper/2103.15670/) (2021)
- [MedViT: A Robust Vision Transformer for Generalized Medical Image Classification](https://mlsanity.com/paper/2302.09462/) (2023)
- [When Adversarial Training Meets Vision Transformers: Recipes from Training to Architecture](https://mlsanity.com/paper/2210.07540/) (2022)
- [Adversarial Attacks Against Medical Deep Learning Systems](https://mlsanity.com/paper/1804.05296/) (2018)
- [Reveal of Vision Transformers Robustness against Adversarial Attacks](https://mlsanity.com/paper/2106.03734/) (2021)
- [A Review of Object Detection Models based on Convolutional Neural Network](https://mlsanity.com/paper/1905.01614/) (2019)
- [Self-Ensembling Vision Transformer (SEViT) for Robust Medical Image Classification](https://mlsanity.com/paper/2208.02851/) (2022)
- [Perceptual Evaluation of Adversarial Attacks for CNN-based Image Classification](https://mlsanity.com/paper/1906.00204/) (2019)
- [Assessing Threat of Adversarial Examples on Deep Neural Networks](https://mlsanity.com/paper/1610.04256/) (2016)
- [Automatically Designing CNN Architectures for Medical Image Segmentation](https://mlsanity.com/paper/1807.07663/) (2018)
- [On enhancing the robustness of Vision Transformers: Defensive Diffusion](https://mlsanity.com/paper/2305.08031/) (2023)

## Built on by (most cited first, 1 of 1)

- [Pulmonary Tuberculosis Edge Diagnosis System Based on MindSpore Framework: Low-cost and High-precision Implementation with Ascend 310 Chip](https://mlsanity.com/paper/2502.14885/) (2025)

## Abstract (arXiv)

Vision Transformer (ViT) is becoming widely popular in automating accurate disease diagnosis in medical imaging owing to its robust self-attention mechanism. However, ViTs remain vulnerable to adversarial attacks that may thwart the diagnosis process by leading it to intentional misclassification of critical disease. In this paper, we propose a novel image classification pipeline, namely, S-E Pipeline, that performs multiple pre-processing steps that allow ViT to be trained on critical features so as to reduce the impact of input perturbations by adversaries. Our method uses a combination of segmentation and image enhancement techniques such as Contrast Limited Adaptive Histogram Equalization (CLAHE), Unsharp Masking (UM), and High-Frequency Emphasis filtering (HFE) as preprocessing steps to identify critical features that remain intact even after adversarial perturbations. The experimental study demonstrates that our novel pipeline helps in reducing the effect of adversarial attacks by 72.22% for the ViT-b32 model and 86.58% for the ViT-l32 model. Furthermore, we have shown an end-to-end deployment of our proposed method on the NVIDIA Jetson Orin Nano board to demonstrate its practical use case in modern hand-held devices that are usually resource-constrained.