# Improving Adversarial Robustness via Mutual Information Estimation

arXiv: https://arxiv.org/abs/2207.12203 · HTML: https://mlsanity.com/paper/2207.12203/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 47 references that are in the map.
- First posted on arXiv 25 Jul 2022.

## Builds on (most cited first, 30 of 32)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Theoretically Principled Trade-off between Robustness and Accuracy](https://mlsanity.com/paper/1901.08573/) (2019)
- [Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks](https://mlsanity.com/paper/2003.01690/) (2020)
- [Learning deep representations by mutual information estimation and maximization](https://mlsanity.com/paper/1808.06670/) (2018)
- [Adversarial Examples Are Not Bugs, They Are Features](https://mlsanity.com/paper/1905.02175/) (2019)
- [MINE: Mutual Information Neural Estimation](https://mlsanity.com/paper/1801.04062/) (2018)
- [Improving Transferability of Adversarial Examples with Input Diversity](https://mlsanity.com/paper/1803.06978/) (2018)
- [Countering Adversarial Images using Input Transformations](https://mlsanity.com/paper/1711.00117/) (2017)
- [Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks](https://mlsanity.com/paper/1704.01155/) (2017)
- [Evading Defenses to Transferable Adversarial Examples by Translation-Invariant Attacks](https://mlsanity.com/paper/1904.02884/) (2019)
- [Defense against Adversarial Attacks Using High-Level Representation Guided Denoiser](https://mlsanity.com/paper/1712.02976/) (2017)
- [Characterizing Adversarial Subspaces Using Local Intrinsic Dimensionality](https://mlsanity.com/paper/1801.02613/) (2018)
- [Spatially Transformed Adversarial Examples](https://mlsanity.com/paper/1801.02612/) (2018)
- [A Self-supervised Approach for Adversarial Robustness](https://mlsanity.com/paper/2006.04924/) (2020)
- [Understanding Adversarial Attacks on Deep Learning Based Medical Image Analysis Systems](https://mlsanity.com/paper/1907.10456/) (2019)
- [Decoupling Direction and Norm for Efficient Gradient-Based L2 Adversarial Attacks and Defenses](https://mlsanity.com/paper/1811.09600/) (2018)
- [Regularizing Neural Networks via Adversarial Model Perturbation](https://mlsanity.com/paper/2010.04925/) (2020)
- [APE-GAN: Adversarial Perturbation Elimination with GAN](https://mlsanity.com/paper/1707.05474/) (2017)
- [Learning Disentangled Representations via Mutual Information Estimation](https://mlsanity.com/paper/1912.03915/) (2019)
- [Fast and Accurate Recurrent Neural Network Acoustic Models for Speech Recognition](https://mlsanity.com/paper/1507.06947/) (2015)
- [Stronger and Faster Wasserstein Adversarial Attacks](https://mlsanity.com/paper/2008.02883/) (2020)
- [Towards Defending against Adversarial Examples via Attack-Invariant Features](https://mlsanity.com/paper/2106.05036/) (2021)
- [On the Sensitivity of Adversarial Robustness to Input Data Distributions](https://mlsanity.com/paper/1902.08336/) (2019)

## Built on by (most cited first, 13 of 13)

- [Multimodal Variational Auto-encoder based Audio-Visual Segmentation](https://mlsanity.com/paper/2310.08303/) (2023)
- [Survey on Adversarial Attack and Defense for Medical Image Analysis: Methods and Challenges](https://mlsanity.com/paper/2303.14133/) (2023)
- [Eliminating Catastrophic Overfitting Via Abnormal Adversarial Examples Regularization](https://mlsanity.com/paper/2404.08154/) (2024)
- [Measure-Theoretic Anti-Causal Representation Learning](https://mlsanity.com/paper/2510.18052/) (2025)
- [Enhancing the "Immunity" of Mixture-of-Experts Networks for Adversarial Defense](https://mlsanity.com/paper/2402.18787/) (2024)
- [Learning Robust and Privacy-Preserving Representations via Information Theory](https://mlsanity.com/paper/2412.11066/) (2024)
- [Why Do Unlearnable Examples Work: A Novel Perspective of Mutual Information](https://mlsanity.com/paper/2603.03725/) (2026)
- [Enhancing 3D Robotic Vision Robustness by Minimizing Adversarial Mutual Information through a Curriculum Training Approach](https://mlsanity.com/paper/2409.12379/) (2024)
- [Adversarially Robust AI-Generated Image Detection for Free: An Information Theoretic Perspective](https://mlsanity.com/paper/2505.22604/) (2025)
- [DBLP: Noise Bridge Consistency Distillation For Efficient And Reliable Adversarial Purification](https://mlsanity.com/paper/2508.00552/) (2025)
- [A Minimax Approach Against Multi-Armed Adversarial Attacks Detection](https://mlsanity.com/paper/2302.02216/) (2023)
- [Designing an attack-defense game: how to increase robustness of financial transaction models via a competition](https://mlsanity.com/paper/2308.11406/) (2023)
- [F2AT: Feature-Focusing Adversarial Training via Disentanglement of Natural and Perturbed Patterns](https://mlsanity.com/paper/2310.14561/) (2023)

## Abstract (arXiv)

Deep neural networks (DNNs) are found to be vulnerable to adversarial noise. They are typically misled by adversarial samples to make wrong predictions. To alleviate this negative effect, in this paper, we investigate the dependence between outputs of the target model and input adversarial samples from the perspective of information theory, and propose an adversarial defense method. Specifically, we first measure the dependence by estimating the mutual information (MI) between outputs and the natural patterns of inputs (called natural MI) and MI between outputs and the adversarial patterns of inputs (called adversarial MI), respectively. We find that adversarial samples usually have larger adversarial MI and smaller natural MI compared with those w.r.t. natural samples. Motivated by this observation, we propose to enhance the adversarial robustness by maximizing the natural MI and minimizing the adversarial MI during the training process. In this way, the target model is expected to pay more attention to the natural pattern that contains objective semantics. Empirical evaluations demonstrate that our method could effectively improve the adversarial accuracy against multiple attacks.