mlsanityData to 7 Oct 2026

Improving Adversarial Robustness via Mutual Information Estimation

arXiv 2207.12203 ↗First posted 25 Jul 2022cs.LGcited by 13

Not on the map. Why?

Abstract (arXiv)

Deep neural networks (DNNs) are found to be vulnerable to adversarial noise. They are typically misled by adversarial samples to make wrong predictions. To alleviate this negative effect, in this paper, we investigate the dependence between outputs of the target model and input adversarial samples from the perspective of information theory, and propose an adversarial defense method. Specifically, we first measure the dependence by estimating the mutual information (MI) between outputs and the natural patterns of inputs (called natural MI) and MI between outputs and the adversarial patterns of inputs (called adversarial MI), respectively. We find that adversarial samples usually have larger adversarial MI and smaller natural MI compared with those w.r.t. natural samples. Motivated by this observation, we propose to enhance the adversarial robustness by maximizing the natural MI and minimizing the adversarial MI during the training process. In this way, the target model is expected to pay more attention to the natural pattern that contains objective semantics. Empirical evaluations demonstrate that our method could effectively improve the adversarial accuracy against multiple attacks.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 32 of its 47 references that are in the map. First posted on arXiv 25 Jul 2022.

Builds on

32 of 47 references.

References · 32

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  3. 2017Mask R-CNNcited by 7,259
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  5. 2013Intriguing properties of neural networkscited by 5,114
  6. 2016Wide Residual Networkscited by 3,057
  7. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  8. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  9. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  10. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
  11. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,265
  12. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 950
20 more
  1. 2018MINE: Mutual Information Neural Estimationcited by 680
  2. 2018Improving Transferability of Adversarial Examples with Input Diversitycited by 591
  3. 2017Countering Adversarial Images using Input Transformationscited by 517
  4. 2017Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networkscited by 484
  5. 2019Evading Defenses to Transferable Adversarial Examples by Translation-Invariant Attackscited by 440
  6. 2017Defense against Adversarial Attacks Using High-Level Representation Guided Denoisercited by 329
  7. 2018Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionalitycited by 280
  8. 2018Spatially Transformed Adversarial Examplescited by 195
  9. 2020A Self-supervised Approach for Adversarial Robustnesscited by 180
  10. 2019Understanding Adversarial Attacks on Deep Learning Based Medical Image Analysis Systemscited by 151
  11. 2018Decoupling Direction and Norm for Efficient Gradient-Based L2 Adversarial Attacks and Defensescited by 109
  12. 2020Regularizing Neural Networks via Adversarial Model Perturbationcited by 78
  13. 2017APE-GAN: Adversarial Perturbation Elimination with GANcited by 57
  14. 2019Learning Disentangled Representations via Mutual Information Estimationcited by 52
  15. 2015Fast and Accurate Recurrent Neural Network Acoustic Models for Speech Recognitioncited by 26
  16. 2020Stronger and Faster Wasserstein Adversarial Attackscited by 25
  17. 2021Towards Defending against Adversarial Examples via Attack-Invariant Featurescited by 25
  18. 2019On the Sensitivity of Adversarial Robustness to Input Data Distributionscited by 20
  19. 2021Removing Adversarial Noise in Class Activation Feature Spacecited by 18
  20. 2020Learning Adversarially Robust Representations via Worst-Case Mutual Information Maximizationcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 4; 2025: 3; 2026: 1.

13 papers.

Sort:

By area:

  1. 2023Multimodal Variational Auto-encoder based Audio-Visual Segmentationcited by 37
  2. 2023Survey on Adversarial Attack and Defense for Medical Image Analysis: Methods and Challengescited by 16
  3. 2024Eliminating Catastrophic Overfitting Via Abnormal Adversarial Examples Regularizationcited by 11
  4. 2024Enhancing the "Immunity" of Mixture-of-Experts Networks for Adversarial Defensecited by 3
  5. 2024Learning Robust and Privacy-Preserving Representations via Information Theorycited by 3
  6. 2025Measure-Theoretic Anti-Causal Representation Learningcited by 3
  7. 2026Why Do Unlearnable Examples Work: A Novel Perspective of Mutual Informationcited by 2
  8. 2024Enhancing 3D Robotic Vision Robustness by Minimizing Adversarial Mutual Information through a Curriculum Training Approachcited by 1
  9. 2025Adversarially Robust AI-Generated Image Detection for Free: An Information Theoretic Perspectivecited by 1
  10. 2025DBLP: Noise Bridge Consistency Distillation For Efficient And Reliable Adversarial Purificationcited by 1
  11. 2023A Minimax Approach Against Multi-Armed Adversarial Attacks Detectioncited by 0
  12. 2023Designing an attack-defense game: how to increase robustness of financial transaction models via a competitioncited by 0
1 more
  1. 2023F2AT: Feature-Focusing Adversarial Training via Disentanglement of Natural and Perturbed Patternscited by 0