mlsanityData to 14 Oct 2026

Quantitative Analysis of Deeply Quantized Tiny Neural Networks Robust to Adversarial Attacks

arXiv 2503.08973 ↗First posted 12 Mar 2025cs.LG cs.CR cs.PFcited by 0

Not on the map. Why?

Abstract (arXiv)

Reducing the memory footprint of Machine Learning (ML) models, especially Deep Neural Networks (DNNs), is imperative to facilitate their deployment on resource-constrained edge devices. However, a notable drawback of DNN models lies in their susceptibility to adversarial attacks, wherein minor input perturbations can deceive them. A primary challenge revolves around the development of accurate, resilient, and compact DNN models suitable for deployment on resource-constrained edge devices. This paper presents the outcomes of a compact DNN model that exhibits resilience against both black-box and white-box adversarial attacks. This work has achieved this resilience through training with the QKeras quantization-aware training framework. The study explores the potential of QKeras and an adversarial robustness technique, Jacobian Regularization (JR), to co-optimize the DNN architecture through per-layer JR methodology. As a result, this paper has devised a DNN model employing this co-optimization strategy based on Stochastic Ternary Quantization (STQ). Its performance was compared against existing DNN models in the face of various white-box and black-box attacks. The experimental findings revealed that, the proposed DNN model had small footprint and on average, it exhibited better performance than Quanos and DS-CNN MLCommons/TinyML (MLC/T) benchmarks when challenged with white-box and black-box attacks, respectively, on the CIFAR-10 image and Google Speech Commands audio datasets.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 34 of its 65 references that are in the map. First posted on arXiv 12 Mar 2025.

Builds on

34 of 65 references.

References · 34

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  3. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  4. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
  5. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 815
  6. 2019Square Attack: a query-efficient black-box adversarial attack via random searchcited by 685
  7. 2017ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Modelscited by 666
  8. 2017One pixel attack for fooling deep neural networkscited by 582
  9. 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 569
  10. 2019Adversarial Attacks and Defenses in Images, Graphs and Text: A Reviewcited by 237
  11. 2018Adversarial Robustness Toolbox v1.0.0cited by 215
  12. 2020TensorFlow Lite Micro: Embedded Machine Learning on TinyML Systemscited by 165
22 more
  1. 2021MLPerf Tiny Benchmarkcited by 124
  2. 2020Tempered Sigmoid Activations for Deep Learning with Differential Privacycited by 122
  3. 2016Robust Large Margin Deep Neural Networkscited by 118
  4. 2019Defensive Quantization: When Efficiency Meets Robustnesscited by 95
  5. 2018Improving DNN Robustness to Adversarial Attacks using Jacobian Regularizationcited by 88
  6. 2020Automatic heterogeneous quantization of deep neural networks for low-latency inference on the edge for particle detectorscited by 80
  7. 2019Detecting Adversarial Samples Using Influence Functions and Nearest Neighborscited by 72
  8. 2020Opportunities and Challenges in Deep Learning Adversarial Robustness: A Surveycited by 72
  9. 2021Random Noise Defense Against Query-Based Black-Box Attackscited by 47
  10. 2021WaveGuard: Understanding and Mitigating Audio Adversarial Examplescited by 40
  11. 2019Depthwise Convolution is All You Need for Learning Multiple Visual Domainscited by 36
  12. 2017Attacking Binarized Neural Networkscited by 29
  13. 2017Learning Accurate Low-Bit Deep Neural Networks with Stochastic Quantizationcited by 21
  14. 2018A comparison of deep networks with ReLU activation function and linear spline-type methodscited by 18
  15. 2020QUANOS- Adversarial Noise Sensitivity Driven Hybrid Quantization of Neural Networkscited by 12
  16. 2020Improving Adversarial Robustness in Weight-quantized Neural Networkscited by 11
  17. 2021Enabling Binary Neural Network Training on the Edgecited by 11
  18. 2021On the Adversarial Robustness of Quantized Neural Networkscited by 11
  19. 2019Impact of Low-bitwidth Quantization on the Adversarial Robustness for Embedded Neural Networkscited by 9
  20. 2020Evaluation of Adversarial Training on Different Types of Neural Networks in Deep Learning-based IDSscited by 9
  21. 2020Bridging the Performance Gap between FGSM and PGD Adversarial Trainingcited by 9
  22. 2023Improving Robustness Against Adversarial Attacks with Deeply Quantized Neural Networkscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2021A Layer-wise Adversarial-aware Quantization Optimization for Improving Robustness
  2. 2023Improving the Robustness of Quantized Deep Neural Networks to White-Box Attacks using Stochastic Quantization and Information-Theoretic Ensemble Training
  3. 2025Breaking the Limits of Quantization-Aware Defenses: QADT-R for Robustness Against Patch-Based Adversarial Attacks in QNNs
  4. 2024David and Goliath: An Empirical Evaluation of Attacks and Defenses for QNNs at the Deep Edge
  5. 2022ODG-Q: Robust Quantization via Online Domain Generalization
  6. 2024Investigating the Impact of Quantization on Adversarial Robustness
  7. 2026RESQ: A Unified Framework for REliability- and Security Enhancement of Quantized Deep Neural Networks
  8. 2021Stochastic-Shield: A Probabilistic Approach Towards Training-Free Adversarial Defense in Quantized CNNs