# What it Thinks is Important is Important: Robustness Transfers through Input Gradients

arXiv: https://arxiv.org/abs/1912.05699 · HTML: https://mlsanity.com/paper/1912.05699/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Dec 2019.

## Built on by (most cited first, 30 of 34)

- [Understanding Zero-Shot Adversarial Robustness for Large-Scale Models](https://mlsanity.com/paper/2212.07016/) (2022)
- [On Fast Adversarial Robustness Adaptation in Model-Agnostic Meta-Learning](https://mlsanity.com/paper/2102.10454/) (2021)
- [Adversarial Robustness for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2109.00946/) (2021)
- [Removing Undesirable Feature Contributions Using Out-of-Distribution Data](https://mlsanity.com/paper/2101.06639/) (2021)
- [Does Robustness on ImageNet Transfer to Downstream Tasks?](https://mlsanity.com/paper/2204.03934/) (2022)
- [On the Versatile Uses of Partial Distance Correlation in Deep Learning](https://mlsanity.com/paper/2207.09684/) (2022)
- [How and When Adversarial Robustness Transfers in Knowledge Distillation?](https://mlsanity.com/paper/2110.12072/) (2021)
- [Do Perceptually Aligned Gradients Imply Adversarial Robustness?](https://mlsanity.com/paper/2207.11378/) (2022)
- [How Does Frequency Bias Affect the Robustness of Neural Image Classifiers against Common Corruption and Adversarial Perturbations?](https://mlsanity.com/paper/2205.04533/) (2022)
- [Federated Robustness Propagation: Sharing Robustness in Heterogeneous Federated Learning](https://mlsanity.com/paper/2106.10196/) (2021)
- [MixACM: Mixup-Based Robustness Transfer via Distillation of Activated Channel Maps](https://mlsanity.com/paper/2111.05073/) (2021)
- [Noise Optimization for Artificial Neural Networks](https://mlsanity.com/paper/2102.04450/) (2021)
- [Transferring Adversarial Robustness Through Robust Representation Matching](https://mlsanity.com/paper/2202.09994/) (2022)
- [A Survey on AI Sustainability: Emerging Trends on Learning Algorithms and Research Challenges](https://mlsanity.com/paper/2205.03824/) (2022)
- [Accelerating Certified Robustness Training via Knowledge Transfer](https://mlsanity.com/paper/2210.14283/) (2022)
- [Indirect Gradient Matching for Adversarial Robust Distillation](https://mlsanity.com/paper/2312.03286/) (2023)
- [Explainable artificial intelligence approaches for brain-computer interfaces: a review and design space](https://mlsanity.com/paper/2312.13033/) (2023)
- [Simple Post-Training Robustness Using Test Time Augmentations and Random Forest](https://mlsanity.com/paper/2109.08191/) (2021)
- [Get Fooled for the Right Reason: Improving Adversarial Robustness through a Teacher-guided Curriculum Learning Approach](https://mlsanity.com/paper/2111.00295/) (2021)
- [Queried Unlabeled Data Improves and Robustifies Class-Incremental Learning](https://mlsanity.com/paper/2206.07842/) (2022)
- [An Eye for an Eye: Defending against Gradient-based Attacks with Gradients](https://mlsanity.com/paper/2202.01117/) (2022)
- [Enhancing Adversarial Robustness via Uncertainty-Aware Distributional Adversarial Training](https://mlsanity.com/paper/2411.02871/) (2024)
- [Inducing Data Amplification Using Auxiliary Datasets in Adversarial Training](https://mlsanity.com/paper/2209.14053/) (2022)
- [Rethinking the Number of Shots in Robust Model-Agnostic Meta-Learning](https://mlsanity.com/paper/2211.15180/) (2022)
- [Advancing Adversarial Training by Injecting Booster Signal](https://mlsanity.com/paper/2306.15451/) (2023)
- [CIARD: Cyclic Iterative Adversarial Robustness Distillation](https://mlsanity.com/paper/2509.12633/) (2025)
- [Fitting Is Not Enough: Smoothness in Extremely Quantized LLMs](https://mlsanity.com/paper/2605.08894/) (2026)
- [Noise Modulation: Let Your Model Interpret Itself](https://mlsanity.com/paper/2103.10603/) (2021)
- [Releasing Inequality Phenomenon in ℓ∞-norm Adversarial Training via Input Gradient Distillation](https://mlsanity.com/paper/2305.09305/) (2023)
- [AdvFunMatch: When Consistent Teaching Meets Adversarial Robustness](https://mlsanity.com/paper/2305.14700/) (2023)

## Abstract (arXiv)

Adversarial perturbations are imperceptible changes to input pixels that can change the prediction of deep learning models. Learned weights of models robust to such perturbations are previously found to be transferable across different tasks but this applies only if the model architecture for the source and target tasks is the same. Input gradients characterize how small changes at each input pixel affect the model output. Using only natural images, we show here that training a student model's input gradients to match those of a robust teacher model can gain robustness close to a strong baseline that is robustly trained from scratch. Through experiments in MNIST, CIFAR-10, CIFAR-100 and Tiny-ImageNet, we show that our proposed method, input gradient adversarial matching, can transfer robustness across different tasks and even across different model architectures. This demonstrates that directly targeting the semantics of input gradients is a feasible way towards adversarial robustness.