# Partitioning the Sample Space for a More Precise Shannon Entropy Estimation

arXiv: https://arxiv.org/abs/2512.10133 · HTML: https://mlsanity.com/paper/2512.10133/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 4 of its 22 references that are in the map.
- First posted on arXiv 10 Dec 2025.

## Builds on (most cited first, 4 of 4)

- [Cross-Entropy Loss Functions: Theoretical Analysis and Applications](https://mlsanity.com/paper/2304.07288/) (2023)
- [Estimating the Entropy of Linguistic Distributions](https://mlsanity.com/paper/2204.01469/) (2022)
- [Data Amplification: A Unified and Competitive Approach to Property Estimation](https://mlsanity.com/paper/1904.00070/) (2019)
- [How Much is Unseen Depends Chiefly on Information About the Seen](https://mlsanity.com/paper/2402.05835/) (2024)

## Abstract (arXiv)

Reliable data-driven estimation of Shannon entropy from small data sets, where the number of examples is potentially smaller than the number of possible outcomes, is a critical matter in several applications. In this paper, we introduce a discrete entropy estimator, where we use the decomposability property in combination with estimations of the missing mass and the number of unseen outcomes to compensate for the negative bias induced by them. Experimental results show that the proposed method outperforms some classical estimators in undersampled regimes, and performs comparably with some well-established state-of-the-art estimators.