# Hold me tight! Influence of discriminative features on deep network boundaries

arXiv: https://arxiv.org/abs/2002.06349 · HTML: https://mlsanity.com/paper/2002.06349/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Feb 2020.

## Built on by (most cited first, 30 of 31)

- [Vision Transformers are Robust Learners](https://mlsanity.com/paper/2105.07581/) (2021)
- [On the effectiveness of adversarial training against common corruptions](https://mlsanity.com/paper/2103.02325/) (2021)
- [Reveal of Vision Transformers Robustness against Adversarial Attacks](https://mlsanity.com/paper/2106.03734/) (2021)
- [Spectral Bias in Practice: The Role of Function Frequency in Generalization](https://mlsanity.com/paper/2110.02424/) (2021)
- [Exploring and Exploiting Decision Boundary Dynamics for Adversarial Robustness](https://mlsanity.com/paper/2302.03015/) (2023)
- [Are socially-aware trajectory prediction models really socially-aware?](https://mlsanity.com/paper/2108.10879/) (2021)
- [A Frequency Perspective of Adversarial Robustness](https://mlsanity.com/paper/2111.00861/) (2021)
- [Removing Undesirable Feature Contributions Using Out-of-Distribution Data](https://mlsanity.com/paper/2101.06639/) (2021)
- [Poisons that are learned faster are more effective](https://mlsanity.com/paper/2204.08615/) (2022)
- [Local Competition and Stochasticity for Adversarial Robustness in Deep Learning](https://mlsanity.com/paper/2101.01121/) (2021)
- [When Deep Classifiers Agree: Analyzing Correlations between Learning Order and Image Statistics](https://mlsanity.com/paper/2105.08997/) (2021)
- [How Does Frequency Bias Affect the Robustness of Neural Image Classifiers against Common Corruption and Adversarial Perturbations?](https://mlsanity.com/paper/2205.04533/) (2022)
- [Deep Learning Generalization and the Convex Hull of Training Sets](https://mlsanity.com/paper/2101.09849/) (2021)
- [LPF-Defense: 3D Adversarial Defense based on Frequency Analysis](https://mlsanity.com/paper/2202.11287/) (2022)
- [It Is All About Data: A Survey on the Effects of Data on Adversarial Robustness](https://mlsanity.com/paper/2303.09767/) (2023)
- [Revealing the Dark Secrets of Extremely Large Kernel ConvNets on Robustness](https://mlsanity.com/paper/2407.08972/) (2024)
- [Understanding Deep Learning via Decision Boundary](https://mlsanity.com/paper/2206.01515/) (2022)
- [CLAD: A Contrastive Learning based Approach for Background Debiasing](https://mlsanity.com/paper/2210.02748/) (2022)
- [Bridging the Gap Between Adversarial Robustness and Optimization Bias](https://mlsanity.com/paper/2102.08868/) (2021)
- [Understanding robustness and generalization of artificial neural networks through Fourier masks](https://mlsanity.com/paper/2203.08822/) (2022)
- [Empirical Advocacy of Bio-inspired Models for Robust Image Recognition](https://mlsanity.com/paper/2205.09037/) (2022)
- [Testing autonomous vehicles and AI: perspectives and challenges from cybersecurity, transparency, robustness and fairness](https://mlsanity.com/paper/2403.14641/) (2024)
- [On the benefits of robust models in modulation recognition](https://mlsanity.com/paper/2103.14977/) (2021)
- [SafeAMC: Adversarial training for robust modulation recognition models](https://mlsanity.com/paper/2105.13746/) (2021)
- [Catastrophic overfitting can be induced with discriminative non-robust features](https://mlsanity.com/paper/2206.08242/) (2022)
- [Models Out of Line: A Fourier Lens on Distribution Shift Robustness](https://mlsanity.com/paper/2207.04075/) (2022)
- [Fourier Sensitivity and Regularization of Computer Vision Models](https://mlsanity.com/paper/2301.13514/) (2023)
- [Frequency-Based Vulnerability Analysis of Deep Learning Models against Image Corruptions](https://mlsanity.com/paper/2306.07178/) (2023)
- [Improving classifier decision boundaries using nearest neighbors](https://mlsanity.com/paper/2310.03927/) (2023)
- [The Vanishing Decision Boundary Complexity and the Strong First Component](https://mlsanity.com/paper/2211.16209/) (2022)

## Abstract (arXiv)

Important insights towards the explainability of neural networks reside in the characteristics of their decision boundaries. In this work, we borrow tools from the field of adversarial robustness, and propose a new perspective that relates dataset features to the distance of samples to the decision boundary. This enables us to carefully tweak the position of the training samples and measure the induced changes on the boundaries of CNNs trained on large-scale vision datasets. We use this framework to reveal some intriguing properties of CNNs. Specifically, we rigorously confirm that neural networks exhibit a high invariance to non-discriminative features, and show that the decision boundaries of a DNN can only exist as long as the classifier is trained with some features that hold them together. Finally, we show that the construction of the decision boundary is extremely sensitive to small perturbations of the training samples, and that changes in certain directions can lead to sudden invariances in the orthogonal ones. This is precisely the mechanism that adversarial training uses to achieve robustness.