# Shannon Entropy Estimation in ∞-Alphabets from Convergence Results

arXiv: https://arxiv.org/abs/1710.06835 · HTML: https://mlsanity.com/paper/1710.06835/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Aug 2017.

## Built on by (most cited first, 1 of 1)

- [Understanding Encoder-Decoder Structures in Machine Learning Using Information Measures](https://mlsanity.com/paper/2405.20452/) (2024)

## Abstract (arXiv)

The problem of Shannon entropy estimation in countable infinite alphabets is addressed from the study and use of convergence results of the entropy functional, which is known to be discontinuous with respect to the total variation distance in ∞-alphabets. Sufficient conditions for the convergence of the entropy are used, including scenarios with both finitely and infinitely supported assumptions on the distributions. From this new perspective, four plug-in histogram-based estimators are studied showing that convergence results are instrumental to derive new strong consistency and rate of convergences results. Different scenarios and conditions are used on both the estimators and the underlying distribution, considering for example finite and unknown supported assumptions and summable tail bounded conditions.