# A Layer-wise Adversarial-aware Quantization Optimization for Improving Robustness

arXiv: https://arxiv.org/abs/2110.12308 · HTML: https://mlsanity.com/paper/2110.12308/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 41 references that are in the map.
- First posted on arXiv 23 Oct 2021.

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples](https://mlsanity.com/paper/1802.00420/) (2018)
- [Adversarial Machine Learning at Scale](https://mlsanity.com/paper/1611.01236/) (2016)
- [Practical Black-Box Attacks against Machine Learning](https://mlsanity.com/paper/1602.02697/) (2016)
- [DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients](https://mlsanity.com/paper/1606.06160/) (2016)
- [Learning Structured Sparsity in Deep Neural Networks](https://mlsanity.com/paper/1608.03665/) (2016)
- [Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples](https://mlsanity.com/paper/1605.07277/) (2016)
- [Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations](https://mlsanity.com/paper/1609.07061/) (2016)
- [Adversarial Examples: Attacks and Defenses for Deep Learning](https://mlsanity.com/paper/1712.07107/) (2017)
- [Defense against Adversarial Attacks Using High-Level Representation Guided Denoiser](https://mlsanity.com/paper/1712.02976/) (2017)
- [Parseval Networks: Improving Robustness to Adversarial Examples](https://mlsanity.com/paper/1704.08847/) (2017)
- [Event-based Vision meets Deep Learning on Steering Prediction for Self-driving Cars](https://mlsanity.com/paper/1804.01310/) (2018)
- [Convolutional Neural Networks using Logarithmic Data Representation](https://mlsanity.com/paper/1603.01025/) (2016)
- [Defensive Quantization: When Efficiency Meets Robustness](https://mlsanity.com/paper/1904.08444/) (2019)
- [Fractional Max-Pooling](https://mlsanity.com/paper/1412.6071/) (2014)
- [Attacking Binarized Neural Networks](https://mlsanity.com/paper/1711.00449/) (2017)
- [Defend Deep Neural Networks Against Adversarial Examples via Fixed and Dynamic Quantized Activation Functions](https://mlsanity.com/paper/1807.06714/) (2018)
- [Quantization of Deep Neural Networks for Accurate Edge Computing](https://mlsanity.com/paper/2104.12046/) (2021)
- [QUANOS- Adversarial Noise Sensitivity Driven Hybrid Quantization of Neural Networks](https://mlsanity.com/paper/2004.11233/) (2020)
- [Evaluating Defensive Distillation For Defending Text Processing Neural Networks Against Adversarial Examples](https://mlsanity.com/paper/1908.07899/) (2019)
- [A Real-Time Deep Learning Pedestrian Detector for Robot Navigation](https://mlsanity.com/paper/1607.04436/) (2016)
- [Feedback Learning for Improving the Robustness of Neural Networks](https://mlsanity.com/paper/1909.05443/) (2019)

## Built on by (most cited first, 3 of 3)

- [A.I. Robustness: a Human-Centered Perspective on Technological Challenges and Opportunities](https://mlsanity.com/paper/2210.08906/) (2022)
- [Boundary-Aware Quantization: Finite-Scale Decision Geometry of Neural Classifiers](https://mlsanity.com/paper/2607.01478/) (2026)
- [Adversarial Attacks on Machine Learning in Embedded and IoT Platforms](https://mlsanity.com/paper/2303.02214/) (2023)

## Abstract (arXiv)

Neural networks are getting better accuracy with higher energy and computational cost. After quantization, the cost can be greatly saved, and the quantized models are more hardware friendly with acceptable accuracy loss. On the other hand, recent research has found that neural networks are vulnerable to adversarial attacks, and the robustness of a neural network model can only be improved with defense methods, such as adversarial training. In this work, we find that adversarially-trained neural networks are more vulnerable to quantization loss than plain models. To minimize both the adversarial and the quantization losses simultaneously and to make the quantized model robust, we propose a layer-wise adversarial-aware quantization method, using the Lipschitz constant to choose the best quantization parameter settings for a neural network. We theoretically derive the losses and prove the consistency of our metric selection. The experiment results show that our method can effectively and efficiently improve the robustness of quantized adversarially-trained neural networks.