# Rarity Determines Minimax Rates of Semantic Entropy From Weak Pairwise Judgments With a Known Channel arXiv: https://arxiv.org/abs/2610.17971 · HTML: https://mlsanity.com/paper/2610.17971/ ## Key facts - No citing papers in the map yet (data through 14 Oct 2026). - Builds on 17 of its 49 references that are in the map. - First posted on arXiv 13 Oct 2026. ## Builds on (most cited first, 17 of 17) - [Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation](https://mlsanity.com/paper/2302.09664/) (2023) - [Generating with Confidence: Uncertainty Quantification for Black-box Large Language Models](https://mlsanity.com/paper/2305.19187/) (2023) - [Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similarities](https://mlsanity.com/paper/2405.20003/) (2024) - [Exact Recovery in the Stochastic Block Model](https://mlsanity.com/paper/1405.3267/) (2014) - [Computational Barriers to Estimation from Low-Degree Polynomials](https://mlsanity.com/paper/2008.02269/) (2020) - [Beyond Semantic Entropy: Boosting LLM Uncertainty Quantification with Pairwise Semantic Similarity](https://mlsanity.com/paper/2506.00245/) (2025) - [Spectrum Estimation from Samples](https://mlsanity.com/paper/1602.00061/) (2016) - [Likelihood landscape and maximum likelihood estimation for the discrete orbit recovery model](https://mlsanity.com/paper/2004.00041/) (2020) - [Estimating Semantic Alphabet Size for LLM Uncertainty Quantification](https://mlsanity.com/paper/2509.14478/) (2025) - [Is it easier to count communities than find them?](https://mlsanity.com/paper/2212.10872/) (2022) - [Hallucination Detection on a Budget: Efficient Bayesian Estimation of Semantic Entropy](https://mlsanity.com/paper/2504.03579/) (2025) - [Estimation of subgraph density in noisy networks](https://mlsanity.com/paper/1803.02488/) (2018) - [Private and Communication-Efficient Algorithms for Entropy Estimation](https://mlsanity.com/paper/2305.07751/) (2023) - [A statistically consistent measure of semantic uncertainty using Language Models](https://mlsanity.com/paper/2502.00507/) (2025) - [Local Poisson Deconvolution for Discrete Signals](https://mlsanity.com/paper/2508.00824/) (2025) - [The generalized method of moments is (almost) statistically efficient in low-SNR Gaussian latent-variable models](https://mlsanity.com/paper/2605.30095/) (2026) - [Characterizing Heterogeneous Rates in Finite Mixture Estimation via Partial Optimal Transport](https://mlsanity.com/paper/2609.16622/) (2026) ## Abstract (arXiv) The semantic entropy of a language model is the entropy of the meanings of its sampled answers. The meanings are never observed, and the entropy is estimated from the verdicts of a pairwise equivalence judge. Given the meanings of n answers, a known channel declares each pair equivalent independently, with probability q+Delta within a meaning and q across meanings. At most K meanings occur, and q is bounded away from 0 and 1. For a weak judge, with nDelta2→0, the global minimax rate is set by distributions near a vertex of the simplex and overstates the difficulty elsewhere. We show that the difficulty is set by how many meanings can be rare. Over distributions with at most s meanings below a fixed probability τ<1/(K-s), the minimax squared risk is of order N-1/s for s≥2, N-1log2(e+N) for s=1 and N-1 for s=0, capped at one, where N=nKDelta2K-2. The matching lower bounds hold on every fixed polynomial window nDelta2≤ n-kappa. One estimator attains these rates without knowing s or τ: it averages products of centered verdicts along all spanning trees of every k-set of answers, projects the resulting power sums onto the valid moment set and inverts them. The case s=K-1 gives the global rate N-1/(K-1) and the consistency threshold: with Delta=n-γ the entropy is uniformly consistently estimable exactly when γ