# Correlation between entropy and generalizability in a neural network

arXiv: https://arxiv.org/abs/2207.01996 · HTML: https://mlsanity.com/paper/2207.01996/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 1 of its 4 references that are in the map.
- First posted on arXiv 5 Jul 2022.

## Builds on (most cited first, 1 of 1)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)

## Abstract (arXiv)

Although neural networks can solve very complex machine-learning problems, the theoretical reason for their generalizability is still not fully understood. Here we use Wang-Landau Mote Carlo algorithm to calculate the entropy (logarithm of the volume of a part of the parameter space) at a given test accuracy, and a given training loss function value or training accuracy. Our results show that entropical forces help generalizability. Although our study is on a very simple application of neural networks (a spiral dataset and a small, fully-connected neural network), our approach should be useful in explaining the generalizability of more complicated neural networks in future works.