Neural Networks Use Distance Metrics
Not on the map. Why?
Abstract (arXiv)
We present empirical evidence that neural networks with ReLU and Absolute Value activations learn distance-based representations. We independently manipulate both distance and intensity properties of internal activations in trained models, finding that both architectures are highly sensitive to small distance-based perturbations while maintaining robust performance under large intensity-based perturbations. These findings challenge the prevailing intensity-based interpretation of neural network activations and offer new insights into their learning and decision-making processes.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 14 of its 31 references that are in the map. First posted on arXiv 26 Nov 2024.
Builds on
14 of 31 references.
References · 14
- 2017Attention Is All You Needcited by 60,361
- 2014Explaining and Harnessing Adversarial Examplescited by 7,364
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
- 2013Intriguing properties of neural networkscited by 5,155
- 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
- 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,849
- 2015Learning Deep Features for Discriminative Localizationcited by 2,436
- 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
- 2013Visualizing and Understanding Convolutional Networkscited by 2,290
- 2014Deep Learning in Neural Networks: An Overviewcited by 770
- 2016The Mythos of Model Interpretabilitycited by 721
- 2017Methods for Interpreting and Understanding Deep Neural Networkscited by 345
2 more
- 2015Understanding Neural Networks Through Deep Visualizationcited by 334
- 2024Interpreting Neural Networks through Mahalanobis Distancecited by 3
Built on by
Citing papers per year: 2025: 1.
1 paper.