mlsanityData to 14 Oct 2026

Improving the Robustness of Quantized Deep Neural Networks to White-Box Attacks using Stochastic Quantization and Information-Theoretic Ensemble Training

arXiv 2312.00105 ↗First posted 30 Nov 2023cs.CV cs.CR cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Most real-world applications that employ deep neural networks (DNNs) quantize them to low precision to reduce the compute needs. We present a method to improve the robustness of quantized DNNs to white-box adversarial attacks. We first tackle the limitation of deterministic quantization to fixed ``bins'' by introducing a differentiable Stochastic Quantizer (SQ). We explore the hypothesis that different quantizations may collectively be more robust than each quantized DNN. We formulate a training objective to encourage different quantized DNNs to learn different representations of the input image. The training objective captures diversity and accuracy via mutual information between ensemble members. Through experimentation, we demonstrate substantial improvement in robustness against L_∞ attacks even if the attacker is allowed to backpropagate through SQ (e.g., > 50% accuracy to PGD(5/255) on CIFAR10 without adversarial training), compared to vanilla DNNs as well as existing ensembles of quantized DNNs. We extend the method to detect attacks and generate robustness profiles in the adversarial information plane (AIP), towards a unified analysis of different threat models by correlating the MI and accuracy.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 34 of its 41 references that are in the map. First posted on arXiv 30 Nov 2023.

Builds on

34 of 41 references.

References · 34

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  4. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,739
  5. 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,485
  6. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,340
  7. 2016Deep Variational Information Bottleneckcited by 930
  8. 2017Ensemble Adversarial Training: Attacks and Defensescited by 840
  9. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 815
  10. 2017Remote Sensing Image Scene Classification: Benchmark and State of the Artcited by 797
  11. 2019Square Attack: a query-efficient black-box adversarial attack via random searchcited by 685
  12. 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 592
22 more
  1. 2017Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methodscited by 546
  2. 2017Detecting Adversarial Samples from Artifactscited by 245
  3. 2017On Detecting Adversarial Perturbationscited by 232
  4. 2019Improving Adversarial Robustness via Promoting Ensemble Diversitycited by 213
  5. 2016Ternary Weight Networkscited by 204
  6. 2018Understanding Measures of Uncertainty for Adversarial Example Detectioncited by 177
  7. 2017Towards Robust Neural Networks via Random Self-ensemblecited by 159
  8. 2019Quantization Networkscited by 153
  9. 2018Scalable Methods for 8-bit Training of Neural Networkscited by 142
  10. 2019Defensive Quantization: When Efficiency Meets Robustnesscited by 95
  11. 2018A Survey on Methods and Theories of Quantized Neural Networkscited by 75
  12. 2017Ensemble Methods as a Defense to Adversarial Perturbations Against Deep Neural Networkscited by 31
  13. 2021PatchGuard++: Efficient Provable Attack Detection against Adversarial Patchescited by 29
  14. 2020EMPIR: Ensembles of Mixed Precision Deep Networks for Increased Robustness against Adversarial Attackscited by 28
  15. 2017Robustness to Adversarial Examples through an Ensemble of Specialistscited by 21
  16. 2018Defend Deep Neural Networks Against Adversarial Examples via Fixed and Dynamic Quantized Activation Functionscited by 17
  17. 2019Information in Infinite Ensembles of Infinitely-Wide Neural Networkscited by 14
  18. 2021Multi-Expert Adversarial Attack Detection in Person Re-identification Using Context Inconsistencycited by 12
  19. 2017Training Ensembles to Detect Adversarial Examplescited by 10
  20. 2019Extracting robust and accurate features via a robust information bottleneckcited by 6
  21. 2018Generalized Ternary Connect: End-to-End Learning and Compression of Multiplication-Free Deep Neural Networkscited by 3
  22. 2020Neural Network Activation Quantization with Bitwise Information Bottleneckscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2021A Layer-wise Adversarial-aware Quantization Optimization for Improving Robustness
  2. 2023Benchmarking the Robustness of Quantized Models
  3. 2022ODG-Q: Robust Quantization via Online Domain Generalization
  4. 2024Investigating the Impact of Quantization on Adversarial Robustness
  5. 2023RobustMQ: Benchmarking Robustness of Quantized Models
  6. 2021Stochastic-Shield: A Probabilistic Approach Towards Training-Free Adversarial Defense in Quantized CNNs
  7. 2025Quantitative Analysis of Deeply Quantized Tiny Neural Networks Robust to Adversarial Attacks
  8. 2023Improving Robustness Against Adversarial Attacks with Deeply Quantized Neural Networks