# Neural Networks Learn Distance Metrics

arXiv: https://arxiv.org/abs/2502.02103 · HTML: https://mlsanity.com/paper/2502.02103/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Feb 2025.

## Built on by (most cited first, 1 of 1)

- [Gradient Descent as Implicit EM in Distance-Based Neural Models](https://mlsanity.com/paper/2512.24780/) (2025)

## Abstract (arXiv)

Neural networks may naturally favor distance-based representations, where smaller activations indicate closer proximity to learned prototypes. This contrasts with intensity-based approaches, which rely on activation magnitudes. To test this hypothesis, we conducted experiments with six MNIST architectural variants constrained to learn either distance or intensity representations. Our results reveal that the underlying representation affects model performance. We develop a novel geometric framework that explains these findings and introduce OffsetL2, a new architecture based on Mahalanobis distance equations, to further validate this framework. This work highlights the importance of considering distance-based learning in neural network design.