# Defend Deep Neural Networks Against Adversarial Examples via Fixed and Dynamic Quantized Activation Functions

arXiv: https://arxiv.org/abs/1807.06714 · HTML: https://mlsanity.com/paper/1807.06714/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jul 2018.

## Built on by (most cited first, 17 of 17)

- [On the Convergence and Robustness of Adversarial Training](https://mlsanity.com/paper/2112.08304/) (2021)
- [Improving Adversarial Robustness via Channel-wise Activation Suppressing](https://mlsanity.com/paper/2103.08307/) (2021)
- [How important are activation functions in regression and classification? A survey, performance comparison, and future directions](https://mlsanity.com/paper/2209.02681/) (2022)
- [Parameterizing Activation Functions for Adversarial Robustness](https://mlsanity.com/paper/2110.05626/) (2021)
- [MeanSparse: Post-Training Robustness Enhancement Through Mean-Centered Feature Sparsification](https://mlsanity.com/paper/2406.05927/) (2024)
- [MIA-Former: Efficient and Robust Vision Transformers via Multi-grained Input-Adaptation](https://mlsanity.com/paper/2112.11542/) (2021)
- [MixACM: Mixup-Based Robustness Transfer via Distillation of Activated Channel Maps](https://mlsanity.com/paper/2111.05073/) (2021)
- [Relationship between Model Compression and Adversarial Robustness: A Review of Current Evidence](https://mlsanity.com/paper/2311.15782/) (2023)
- [2-in-1 Accelerator: Enabling Random Precision Switch for Winning Both Adversarial Robustness and Efficiency](https://mlsanity.com/paper/2109.05223/) (2021)
- [Robust Binary Models by Pruning Randomly-initialized Networks](https://mlsanity.com/paper/2202.01341/) (2022)
- [Tensor Normalization and Full Distribution Training](https://mlsanity.com/paper/2109.02345/) (2021)
- [A Layer-wise Adversarial-aware Quantization Optimization for Improving Robustness](https://mlsanity.com/paper/2110.12308/) (2021)
- [Can collaborative learning be private, robust and scalable?](https://mlsanity.com/paper/2205.02652/) (2022)
- [VQUNet: Vector Quantization U-Net for Defending Adversarial Atacks by Regularizing Unwanted Noise](https://mlsanity.com/paper/2406.03117/) (2024)
- [Improving the Robustness of Quantized Deep Neural Networks to White-Box Attacks using Stochastic Quantization and Information-Theoretic Ensemble Training](https://mlsanity.com/paper/2312.00105/) (2023)
- [Is ReLU Adversarially Robust?](https://mlsanity.com/paper/2405.03777/) (2024)
- [Protecting the Neural Networks against FGSM Attack Using Machine Unlearning](https://mlsanity.com/paper/2511.01377/) (2025)

## Abstract (arXiv)

Recent studies have shown that deep neural networks (DNNs) are vulnerable to adversarial attacks. To this end, many defense approaches that attempt to improve the robustness of DNNs have been proposed. In a separate and yet related area, recent works have explored to quantize neural network weights and activation functions into low bit-width to compress model size and reduce computational complexity. In this work, we find that these two different tracks, namely the pursuit of network compactness and robustness, can be merged into one and give rise to networks of both advantages. To the best of our knowledge, this is the first work that uses quantization of activation functions to defend against adversarial examples. We also propose to train robust neural networks by using adaptive quantization techniques for the activation functions. Our proposed Dynamic Quantized Activation (DQA) is verified through a wide range of experiments with the MNIST and CIFAR-10 datasets under different white-box attack methods, including FGSM, PGD, and C & W attacks. Furthermore, Zeroth Order Optimization and substitute model-based black-box attacks are also considered in this work. The experimental results clearly show that the robustness of DNNs could be greatly improved using the proposed DQA.