# Ratio Covers of Convex Sets and Optimal Mixture Density Estimation

arXiv: https://arxiv.org/abs/2602.16142 · HTML: https://mlsanity.com/paper/2602.16142/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 60 references that are in the map.
- First posted on arXiv 18 Feb 2026.

## Builds on (most cited first, 10 of 10)

- [Learning with Square Loss: Localization through Offset Rademacher Complexity](https://mlsanity.com/paper/1502.06134/) (2015)
- [An improper estimator with optimal excess risk in misspecified density estimation and logistic regression](https://mlsanity.com/paper/1912.10784/) (2019)
- [Near-Optimal Learning of Tree-Structured Distributions by Chow-Liu](https://mlsanity.com/paper/2011.04144/) (2020)
- [High-Probability Risk Bounds via Sequential Predictors](https://mlsanity.com/paper/2308.07588/) (2023)
- [Localization, Convexity, and Star Aggregation](https://mlsanity.com/paper/2105.08866/) (2021)
- [Concentration Bounds for Discrete Distribution Estimation in KL Divergence](https://mlsanity.com/paper/2302.06869/) (2023)
- [Estimation of discrete distributions in relative entropy, and the deviations of the missing mass](https://mlsanity.com/paper/2504.21787/) (2025)
- [Nearly Minimax Discrete Distribution Estimation in Kullback-Leibler Divergence with High Probability](https://mlsanity.com/paper/2507.17316/) (2025)
- [Local Risk Bounds for Statistical Aggregation](https://mlsanity.com/paper/2306.17151/) (2023)
- [How good is the Chord algorithm?](https://mlsanity.com/paper/1309.7084/) (2013)

## Built on by (most cited first, 3 of 3)

- [Hedging on the Frontier: Learning New Tasks with Few Samples](https://mlsanity.com/paper/2605.30997/) (2026)
- [Density estimation for Hellinger via minimum-distance estimators: mixtures of Gaussians, log-concave, and more](https://mlsanity.com/paper/2606.11469/) (2026)
- [Algorithms for adaptive and heteroskedastic linear regression at the computational threshold](https://mlsanity.com/paper/2608.18402/) (2026)

## Abstract (arXiv)

We study density estimation in Kullback-Leibler divergence: given an i.i.d. sample from an unknown density p^star, the goal is to construct an estimator widehatp such that KL(p^star,widehatp) is small with high probability. We consider two fundamental settings involving a finite dictionary of densities: (i) model aggregation, where p^star belongs to the dictionary, and (ii) convex aggregation (mixture density estimation), where p^star is a mixture of densities from the dictionary. Crucially, we make no assumption on the base densities: their ratios may be unbounded and their supports may differ. For both problems, we identify the best possible high-probability guarantees in terms of the dictionary size, sample size, and confidence level. These optimal rates are higher than those achievable when density ratios are bounded by absolute constants; for mixture density estimation, they match existing lower bounds in the special case of discrete distributions. Our analysis of the mixture case hinges on two new covering results. First, we provide a sharp, distribution-free upper bound on the local Hellinger entropy of the class of mixtures of M distributions. Second, we prove an optimal ratio covering theorem for convex sets: for every convex compact set K subset ℝ_+d, there exists a subset A subset K with at most 2O(d) elements such that each element of K is coordinate-wise dominated by an element of A up to a universal constant factor. This geometric result is of independent interest; notably, it yields new cardinality estimates for varepsilon-approximate Pareto sets in multi-objective optimization with convex feasible set.