# Improving Robustness Against Adversarial Attacks with Deeply Quantized Neural Networks

arXiv: https://arxiv.org/abs/2304.12829 · HTML: https://mlsanity.com/paper/2304.12829/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 57 references that are in the map.
- First posted on arXiv 25 Apr 2023.

## Builds on (most cited first, 30 of 34)

- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [A Survey of Quantization Methods for Efficient Neural Network Inference](https://mlsanity.com/paper/2103.13630/) (2021)
- [Square Attack: a query-efficient black-box adversarial attack via random search](https://mlsanity.com/paper/1912.00049/) (2019)
- [ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Models](https://mlsanity.com/paper/1708.03999/) (2017)
- [One pixel attack for fooling deep neural networks](https://mlsanity.com/paper/1710.08864/) (2017)
- [DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients](https://mlsanity.com/paper/1606.06160/) (2016)
- [Adversarial Attacks and Defenses in Images, Graphs and Text: A Review](https://mlsanity.com/paper/1909.08072/) (2019)
- [Adversarial Robustness Toolbox v1.0.0](https://mlsanity.com/paper/1807.01069/) (2018)
- [TensorFlow Lite Micro: Embedded Machine Learning on TinyML Systems](https://mlsanity.com/paper/2010.08678/) (2020)
- [MLPerf Tiny Benchmark](https://mlsanity.com/paper/2106.07597/) (2021)
- [Tempered Sigmoid Activations for Deep Learning with Differential Privacy](https://mlsanity.com/paper/2007.14191/) (2020)
- [Robust Large Margin Deep Neural Networks](https://mlsanity.com/paper/1605.08254/) (2016)
- [QEBA: Query-Efficient Boundary-Based Blackbox Attack](https://mlsanity.com/paper/2005.14137/) (2020)
- [Defensive Quantization: When Efficiency Meets Robustness](https://mlsanity.com/paper/1904.08444/) (2019)
- [Improving DNN Robustness to Adversarial Attacks using Jacobian Regularization](https://mlsanity.com/paper/1803.08680/) (2018)
- [Automatic heterogeneous quantization of deep neural networks for low-latency inference on the edge for particle detectors](https://mlsanity.com/paper/2006.10159/) (2020)
- [Detecting Adversarial Samples Using Influence Functions and Nearest Neighbors](https://mlsanity.com/paper/1909.06872/) (2019)
- [Opportunities and Challenges in Deep Learning Adversarial Robustness: A Survey](https://mlsanity.com/paper/2007.00753/) (2020)
- [Random Noise Defense Against Query-Based Black-Box Attacks](https://mlsanity.com/paper/2104.11470/) (2021)
- [WaveGuard: Understanding and Mitigating Audio Adversarial Examples](https://mlsanity.com/paper/2103.03344/) (2021)
- [Depthwise Convolution is All You Need for Learning Multiple Visual Domains](https://mlsanity.com/paper/1902.00927/) (2019)
- [Attacking Binarized Neural Networks](https://mlsanity.com/paper/1711.00449/) (2017)
- [Learning Accurate Low-Bit Deep Neural Networks with Stochastic Quantization](https://mlsanity.com/paper/1708.01001/) (2017)
- [Adversarial Robustness via Fisher-Rao Regularization](https://mlsanity.com/paper/2106.06685/) (2021)
- [A comparison of deep networks with ReLU activation function and linear spline-type methods](https://mlsanity.com/paper/1804.02253/) (2018)
- [QUANOS- Adversarial Noise Sensitivity Driven Hybrid Quantization of Neural Networks](https://mlsanity.com/paper/2004.11233/) (2020)
- [Improving Adversarial Robustness in Weight-quantized Neural Networks](https://mlsanity.com/paper/2012.14965/) (2020)
- [Enabling Binary Neural Network Training on the Edge](https://mlsanity.com/paper/2102.04270/) (2021)
- [On the Adversarial Robustness of Quantized Neural Networks](https://mlsanity.com/paper/2105.00227/) (2021)

## Built on by (most cited first, 3 of 3)

- [DeltaNN: Assessing the Impact of Computational Environment Parameters on the Performance of Image Recognition Models](https://mlsanity.com/paper/2306.06208/) (2023)
- [Securing Time Integrity in Energy IoT Against Clock Drift and Y2K38 Failures](https://mlsanity.com/paper/2601.23147/) (2026)
- [Quantitative Analysis of Deeply Quantized Tiny Neural Networks Robust to Adversarial Attacks](https://mlsanity.com/paper/2503.08973/) (2025)

## Abstract (arXiv)

Reducing the memory footprint of Machine Learning (ML) models, particularly Deep Neural Networks (DNNs), is essential to enable their deployment into resource-constrained tiny devices. However, a disadvantage of DNN models is their vulnerability to adversarial attacks, as they can be fooled by adding slight perturbations to the inputs. Therefore, the challenge is how to create accurate, robust, and tiny DNN models deployable on resource-constrained embedded devices. This paper reports the results of devising a tiny DNN model, robust to adversarial black and white box attacks, trained with an automatic quantizationaware training framework, i.e. QKeras, with deep quantization loss accounted in the learning loop, thereby making the designed DNNs more accurate for deployment on tiny devices. We investigated how QKeras and an adversarial robustness technique, Jacobian Regularization (JR), can provide a co-optimization strategy by exploiting the DNN topology and the per layer JR approach to produce robust yet tiny deeply quantized DNN models. As a result, a new DNN model implementing this cooptimization strategy was conceived, developed and tested on three datasets containing both images and audio inputs, as well as compared its performance with existing benchmarks against various white-box and black-box attacks. Experimental results demonstrated that on average our proposed DNN model resulted in 8.3% and 79.5% higher accuracy than MLCommons/Tiny benchmarks in the presence of white-box and black-box attacks on the CIFAR-10 image dataset and a subset of the Google Speech Commands audio dataset respectively. It was also 6.5% more accurate for black-box attacks on the SVHN image dataset.