# Visualising Basins of Attraction for the Cross-Entropy and the Squared Error Neural Network Loss Functions

arXiv: https://arxiv.org/abs/1901.02302 · HTML: https://mlsanity.com/paper/1901.02302/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Jan 2019.

## Built on by (most cited first, 13 of 13)

- [Benign Overfitting in Multiclass Classification: All Roads Lead to Interpolation](https://mlsanity.com/paper/2106.10865/) (2021)
- [Generalization for multiclass classification with overparameterized linear models](https://mlsanity.com/paper/2206.01399/) (2022)
- [Dynamical stability and chaos in artificial neural network trajectories along training](https://mlsanity.com/paper/2404.05782/) (2024)
- [Learning from Very Little Data: On the Value of Landscape Analysis for Predicting Software Project Health](https://mlsanity.com/paper/2301.06577/) (2023)
- [Empirical Loss Landscape Analysis of Neural Network Activation Functions](https://mlsanity.com/paper/2306.16090/) (2023)
- [Hybridised Loss Functions for Improved Neural Network Generalisation](https://mlsanity.com/paper/2204.12244/) (2022)
- [Evaluating the Impact of Loss Function Variation in Deep Learning for Classification](https://mlsanity.com/paper/2210.16003/) (2022)
- [Introducing One Sided Margin Loss for Solving Classification Problems in Deep Networks](https://mlsanity.com/paper/2206.01002/) (2022)
- [Physics Inspired Approaches To Understanding Gaussian Processes](https://mlsanity.com/paper/2305.10748/) (2023)
- [Beyond Single-Model Views for Deep Learning: Optimization versus Generalizability of Stochastic Optimization Algorithms](https://mlsanity.com/paper/2403.00574/) (2024)
- [Regularisation in neural networks: a survey and empirical analysis of approaches](https://mlsanity.com/paper/2601.23131/) (2026)
- [Scalar Representations of Neural Network Training Dynamics](https://mlsanity.com/paper/2606.30384/) (2026)
- [Beyond Quadratic Loss: The Stability Phase Diagram of Adam](https://mlsanity.com/paper/2609.18314/) (2026)

## Abstract (arXiv)

Quantification of the stationary points and the associated basins of attraction of neural network loss surfaces is an important step towards a better understanding of neural network loss surfaces at large. This work proposes a novel method to visualise basins of attraction together with the associated stationary points via gradient-based random sampling. The proposed technique is used to perform an empirical study of the loss surfaces generated by two different error metrics: quadratic loss and entropic loss. The empirical observations confirm the theoretical hypothesis regarding the nature of neural network attraction basins. Entropic loss is shown to exhibit stronger gradients and fewer stationary points than quadratic loss, indicating that entropic loss has a more searchable landscape. Quadratic loss is shown to be more resilient to overfitting than entropic loss. Both losses are shown to exhibit local minima, but the number of local minima is shown to decrease with an increase in dimensionality. Thus, the proposed visualisation technique successfully captures the local minima properties exhibited by the neural network loss surfaces, and can be used for the purpose of fitness landscape analysis of neural networks.