# Demystifying Causal Features on Adversarial Examples and Causal Inoculation for Robust Network by Adversarial Instrumental Variable Regression

arXiv: https://arxiv.org/abs/2303.01052 · HTML: https://mlsanity.com/paper/2303.01052/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 81 references that are in the map.
- First posted on arXiv 2 Mar 2023.

## Builds on (most cited first, 30 of 32)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Theoretically Principled Trade-off between Robustness and Accuracy](https://mlsanity.com/paper/1901.08573/) (2019)
- [Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks](https://mlsanity.com/paper/2003.01690/) (2020)
- [Adversarial Examples Are Not Bugs, They Are Features](https://mlsanity.com/paper/1905.02175/) (2019)
- [Robustness May Be at Odds with Accuracy](https://mlsanity.com/paper/1805.12152/) (2018)
- [Fast is better than free: Revisiting adversarial training](https://mlsanity.com/paper/2001.03994/) (2020)
- [Cyclical Learning Rates for Training Neural Networks](https://mlsanity.com/paper/1506.01186/) (2015)
- [Overfitting in adversarially robust deep learning](https://mlsanity.com/paper/2002.11569/) (2020)
- [Understanding Deep Image Representations by Inverting Them](https://mlsanity.com/paper/1412.0035/) (2014)
- [A Fourier Perspective on Model Robustness in Computer Vision](https://mlsanity.com/paper/1906.08988/) (2019)
- [Rademacher Complexity for Adversarially Robust Generalization](https://mlsanity.com/paper/1810.11914/) (2018)
- [Kernel Instrumental Variable Regression](https://mlsanity.com/paper/1906.00232/) (2019)
- [Are adversarial examples inevitable?](https://mlsanity.com/paper/1809.02104/) (2018)
- [On the Power of Over-parametrization in Neural Networks with Quadratic Activation](https://mlsanity.com/paper/1803.01206/) (2018)
- [Deep Generalized Method of Moments for Instrumental Variable Analysis](https://mlsanity.com/paper/1905.12495/) (2019)
- [Minimax Estimation of Conditional Moment Models](https://mlsanity.com/paper/2006.07201/) (2020)
- [Data-dependent Sample Complexity of Deep Neural Networks via Lipschitz Augmentation](https://mlsanity.com/paper/1905.03684/) (2019)
- [Understanding Neural Networks via Feature Visualization: A survey](https://mlsanity.com/paper/1904.08939/) (2019)
- [Discovering and Explaining the Representation Bottleneck of DNNs](https://mlsanity.com/paper/2111.06236/) (2021)
- [Adversarial Generalized Method of Moments](https://mlsanity.com/paper/1803.07164/) (2018)
- [Optimal Sup-norm Rates and Uniform Inference on Nonlinear Functionals of Nonparametric IV Regression](https://mlsanity.com/paper/1508.03365/) (2015)
- [Distilling Robust and Non-Robust Features in Adversarial Examples by Information Bottleneck](https://mlsanity.com/paper/2204.02735/) (2022)
- [Masking Adversarial Damage: Finding Adversarial Saliency for Robust and Sparse Network](https://mlsanity.com/paper/2204.02738/) (2022)

## Built on by (most cited first, 18 of 18)

- [MoAI: Mixture of All Intelligence for Large Language and Vision Models](https://mlsanity.com/paper/2403.07508/) (2024)
- [Meteor: Mamba-based Traversal of Rationale for Large Language and Vision Models](https://mlsanity.com/paper/2405.15574/) (2024)
- [CoLLaVO: Crayon Large Language and Vision mOdel](https://mlsanity.com/paper/2402.11248/) (2024)
- [Causal Unsupervised Semantic Segmentation](https://mlsanity.com/paper/2310.07379/) (2023)
- [Phantom of Latent for Large Language and Vision Models](https://mlsanity.com/paper/2409.14713/) (2024)
- [TroL: Traversal of Layers for Large Language and Vision Models](https://mlsanity.com/paper/2406.12246/) (2024)
- [Mitigating Adversarial Vulnerability through Causal Parameter Estimation by Adversarial Double Machine Learning](https://mlsanity.com/paper/2307.07250/) (2023)
- [GenRecal: Generation after Recalibration from Large to Small Vision-Language Models](https://mlsanity.com/paper/2506.15681/) (2025)
- [CausalDiff: Causality-Inspired Disentanglement via Diffusion Model for Adversarial Defense](https://mlsanity.com/paper/2410.23091/) (2024)
- [Image Quality Assessment: Investigating Causal Perceptual Effects with Abductive Counterfactual Inference](https://mlsanity.com/paper/2412.16939/) (2024)
- [Robust Proxy: Improving Adversarial Robustness by Robust Proxy Learning](https://mlsanity.com/paper/2306.15457/) (2023)
- [Intriguing Properties of Large Language and Vision Models](https://mlsanity.com/paper/2410.04751/) (2024)
- [Weakly Supervised Contrastive Adversarial Training for Learning Robust Features from Semi-supervised Data](https://mlsanity.com/paper/2503.11032/) (2025)
- [Knowledge-Guided Adversarial Training for Infrared Object Detection via Thermal Radiation Modeling](https://mlsanity.com/paper/2603.25170/) (2026)
- [Mid-Harness: Scaling Actions Between Model and Harness for Terminal Agents](https://mlsanity.com/paper/2609.39982/) (2026)
- [Unraveling Attacks in Machine Learning-based IoT Ecosystems: A Survey and the Open Libraries Behind Them](https://mlsanity.com/paper/2401.11723/) (2024)
- [Causal Perception Inspired Representation Learning for Trustworthy Image Quality Assessment](https://mlsanity.com/paper/2404.19567/) (2024)
- [Sustainable Self-evolution Adversarial Training](https://mlsanity.com/paper/2412.02270/) (2024)

## Abstract (arXiv)

The origin of adversarial examples is still inexplicable in research fields, and it arouses arguments from various viewpoints, albeit comprehensive investigations. In this paper, we propose a way of delving into the unexpected vulnerability in adversarially trained networks from a causal perspective, namely adversarial instrumental variable (IV) regression. By deploying it, we estimate the causal relation of adversarial prediction under an unbiased environment dissociated from unknown confounders. Our approach aims to demystify inherent causal features on adversarial examples by leveraging a zero-sum optimization game between a casual feature estimator (i.e., hypothesis model) and worst-case counterfactuals (i.e., test function) disturbing to find causal features. Through extensive analyses, we demonstrate that the estimated causal features are highly related to the correct prediction for adversarial robustness, and the counterfactuals exhibit extreme features significantly deviating from the correct prediction. In addition, we present how to effectively inoculate CAusal FEatures (CAFE) into defense networks for improving adversarial robustness.