# Adversarial Defense via Data Dependent Activation Function and Total Variation Minimization

arXiv: https://arxiv.org/abs/1809.08516 · HTML: https://mlsanity.com/paper/1809.08516/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Sep 2018.

## Built on by (most cited first, 9 of 9)

- [Improving Adversarial Robustness via Channel-wise Activation Suppressing](https://mlsanity.com/paper/2103.08307/) (2021)
- [Parameterizing Activation Functions for Adversarial Robustness](https://mlsanity.com/paper/2110.05626/) (2021)
- [MeanSparse: Post-Training Robustness Enhancement Through Mean-Centered Feature Sparsification](https://mlsanity.com/paper/2406.05927/) (2024)
- [How Does Momentum Benefit Deep Neural Networks Architecture Design? A Few Case Studies](https://mlsanity.com/paper/2110.07034/) (2021)
- [MixACM: Mixup-Based Robustness Transfer via Distillation of Activated Channel Maps](https://mlsanity.com/paper/2111.05073/) (2021)
- [Generalization Error Analysis of Neural networks with Gradient Based Regularization](https://mlsanity.com/paper/2107.02797/) (2021)
- [Adversarial Parameter Defense by Multi-Step Risk Minimization](https://mlsanity.com/paper/2109.02889/) (2021)
- [CausAdv: A Causal-based Framework for Detecting Adversarial Examples](https://mlsanity.com/paper/2411.00839/) (2024)
- [GReAT: A Graph Regularized Adversarial Training Method](https://mlsanity.com/paper/2310.05336/) (2023)

## Abstract (arXiv)

We improve the robustness of Deep Neural Net (DNN) to adversarial attacks by using an interpolating function as the output activation. This data-dependent activation remarkably improves both the generalization and robustness of DNN. In the CIFAR10 benchmark, we raise the robust accuracy of the adversarially trained ResNet20 from ~ 46% to ~ 69% under the state-of-the-art Iterative Fast Gradient Sign Method (IFGSM) based adversarial attack. When we combine this data-dependent activation with total variation minimization on adversarial images and training data augmentation, we achieve an improvement in robust accuracy by 38.9% for ResNet56 under the strongest IFGSM attack. Furthermore, We provide an intuitive explanation of our defense by analyzing the geometry of the feature space.