# The Shannon Entropy of a Histogram

arXiv: https://arxiv.org/abs/2210.02848 · HTML: https://mlsanity.com/paper/2210.02848/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Oct 2022.

## Built on by (most cited first, 3 of 3)

- [Can bin-wise scaling improve consistency and adaptivity of prediction uncertainty for machine learning regression ?](https://mlsanity.com/paper/2310.11978/) (2023)
- [An information theoretic limit to data amplification](https://mlsanity.com/paper/2412.18041/) (2024)
- [Limits to classification performance by relating Kullback-Leibler divergence to Cohen's Kappa](https://mlsanity.com/paper/2403.01571/) (2024)

## Abstract (arXiv)

The histogram is a key method for visualizing data and estimating the underlying probability distribution. Incorrect conclusions about the data result from over or under-binning. A new method based on the Shannon entropy of the histogram uses a simple formula based on the differential entropy estimated from nearest-neighbour distances. Links are made between the new method and other algorithms such as Scott's formula, and cost and risk function methods. A parameter is found that predicts over and under-binning, which can be estimated for any histogram. The new algorithm is shown to be robust by application to real data.