# Improving classifier decision boundaries using nearest neighbors

arXiv: https://arxiv.org/abs/2310.03927 · HTML: https://mlsanity.com/paper/2310.03927/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 39 references that are in the map.
- First posted on arXiv 5 Oct 2023.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [Adversarial Machine Learning at Scale](https://mlsanity.com/paper/1611.01236/) (2016)
- [Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images](https://mlsanity.com/paper/1412.1897/) (2014)
- [A Topological Filter for Learning with Label Noise](https://mlsanity.com/paper/2012.04835/) (2020)
- [Characterizing the Decision Boundary of Deep Neural Networks](https://mlsanity.com/paper/1912.11460/) (2019)
- [Hold me tight! Influence of discriminative features on deep network boundaries](https://mlsanity.com/paper/2002.06349/) (2020)
- [Boundary thickness and robustness in learning models](https://mlsanity.com/paper/2007.05086/) (2020)
- [Robustness to Label Noise Depends on the Shape of the Noise Distribution in Feature Space](https://mlsanity.com/paper/2206.01106/) (2022)
- [On the Decision Boundary of Deep Neural Networks](https://mlsanity.com/paper/1808.05385/) (2018)
- [Explaining Classifiers by Constructing Familiar Concepts](https://mlsanity.com/paper/2203.04109/) (2022)
- [Understanding Deep Learning via Decision Boundary](https://mlsanity.com/paper/2206.01515/) (2022)
- [Classification regions of deep neural networks](https://mlsanity.com/paper/1705.09552/) (2017)
- [Input margins can predict generalization too](https://mlsanity.com/paper/2308.15466/) (2023)

## Built on by (most cited first, 1 of 1)

- [Fast Leave-One-Out Approximation from Fragment-Target Prevalence Vectors (molFTP) : From Dummy Masking to Key-LOO for Leakage-Free Feature Construction](https://mlsanity.com/paper/2510.06029/) (2025)

## Abstract (arXiv)

Neural networks are not learning optimal decision boundaries. We show that decision boundaries are situated in areas of low training data density. They are impacted by few training samples which can easily lead to overfitting. We provide a simple algorithm performing a weighted average of the prediction of a sample and its nearest neighbors' (computed in latent space) leading to a minor favorable outcomes for a variety of important measures for neural networks. In our evaluation, we employ various self-trained and pre-trained convolutional neural networks to show that our approach improves (i) resistance to label noise, (ii) robustness against adversarial attacks, (iii) classification accuracy, and to some degree even (iv) interpretability. While improvements are not necessarily large in all four areas, our approach is conceptually simple, i.e., improvements come without any modification to network architecture, training procedure or dataset. Furthermore, they are in stark contrast to prior works that often require trade-offs among the four objectives or provide valuable, but non-actionable insights.