# Understanding the Decision Boundary of Deep Neural Networks: An Empirical Study

arXiv: https://arxiv.org/abs/2002.01810 · HTML: https://mlsanity.com/paper/2002.01810/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Feb 2020.

## Built on by (most cited first, 20 of 20)

- [Latent Imitator: Generating Natural Individual Discriminatory Instances for Black-Box Fairness Testing](https://mlsanity.com/paper/2305.11602/) (2023)
- [Copyright Protection for Large Language Models: A Survey of Methods, Challenges, and Trends](https://mlsanity.com/paper/2508.11548/) (2025)
- [Understanding CNN Fragility When Learning With Imbalanced Data](https://mlsanity.com/paper/2210.09465/) (2022)
- [Detecting Brittle Decisions for Free: Leveraging Margin Consistency in Deep Robust Classifiers](https://mlsanity.com/paper/2406.18451/) (2024)
- [MANO: Exploiting Matrix Norm for Unsupervised Accuracy Estimation Under Distribution Shifts](https://mlsanity.com/paper/2405.18979/) (2024)
- [Understanding Deep Learning via Decision Boundary](https://mlsanity.com/paper/2206.01515/) (2022)
- [Querying Easily Flip-flopped Samples for Deep Active Learning](https://mlsanity.com/paper/2401.09787/) (2024)
- [Investigating the Corruption Robustness of Image Classifiers with Random Lp-norm Corruptions](https://mlsanity.com/paper/2305.05400/) (2023)
- [BOOD: Boundary-based Out-Of-Distribution Data Generation](https://mlsanity.com/paper/2508.00350/) (2025)
- [Utilizing Class Separation Distance for the Evaluation of Corruption Robustness of Machine Learning Classifiers](https://mlsanity.com/paper/2206.13405/) (2022)
- [Boundary-Aware Quantization: Finite-Scale Decision Geometry of Neural Classifiers](https://mlsanity.com/paper/2607.01478/) (2026)
- [How to Explain Neural Networks: an Approximation Perspective](https://mlsanity.com/paper/2105.07831/) (2021)
- [Towards the Desirable Decision Boundary by Moderate-Margin Adversarial Training](https://mlsanity.com/paper/2207.07793/) (2022)
- [Not So Robust After All: Evaluating the Robustness of Deep Neural Networks to Unseen Adversarial Attacks](https://mlsanity.com/paper/2308.06467/) (2023)
- [Exploring and Leveraging Class Vectors for Classifier Editing](https://mlsanity.com/paper/2510.11268/) (2025)
- [The Vanishing Decision Boundary Complexity and the Strong First Component](https://mlsanity.com/paper/2211.16209/) (2022)
- [MedISure: Towards Assuring Machine Learning-based Medical Image Classifiers using Mixup Boundary Analysis](https://mlsanity.com/paper/2311.13978/) (2023)
- [A Personalized Video-Based Hand Taxonomy: Application for Individuals with Spinal Cord Injury](https://mlsanity.com/paper/2403.18094/) (2024)
- [Decision Potential Surface: A Theoretical and Practical Approximation of Large Language Model Decision Boundary](https://mlsanity.com/paper/2510.03271/) (2025)
- [Margin-Consistent Deep Subtyping of Invasive Lung Adenocarcinoma via Perturbation Fidelity in Whole-Slide Image Analysis](https://mlsanity.com/paper/2603.06650/) (2026)

## Abstract (arXiv)

Despite achieving remarkable performance on many image classification tasks, state-of-the-art machine learning (ML) classifiers remain vulnerable to small input perturbations. Especially, the existence of adversarial examples raises concerns about the deployment of ML models in safety- and security-critical environments, like autonomous driving and disease detection. Over the last few years, numerous defense methods have been published with the goal of improving adversarial as well as corruption robustness. However, the proposed measures succeeded only to a very limited extent. This limited progress is partly due to the lack of understanding of the decision boundary and decision regions of deep neural networks. Therefore, we study the minimum distance of data points to the decision boundary and how this margin evolves over the training of a deep neural network. By conducting experiments on MNIST, FASHION-MNIST, and CIFAR-10, we observe that the decision boundary moves closer to natural images over training. This phenomenon even remains intact in the late epochs of training, where the classifier already obtains low training and test error rates. On the other hand, adversarial training appears to have the potential to prevent this undesired convergence of the decision boundary.