Statistical-Computational Trade-offs for Density Estimation

We study the density estimation problem defined as follows: given k distributions p1, ldots, pk over a discrete domain [n], as well as a collection of samples chosen from a ``query'' distribution q over [n], output pi that is ``close'' to q. Recently~citeaamand2023data gave the first and only known result that achieves sublinear bounds in em both the sampling complexity and the query time while preserving polynomial data structure space. However, their improvement over linear samples and time is only by subpolynomial factors. Our main result is a lower bound showing that, for a broad class of data structures, their bounds cannot be significantly improved. In particular, if an algorithm uses O(n/logc k) samples for some constant c>0 and polynomial space, then the query time of the data structure must be at least k1-O(1)/log log k, i.e., close to linear in the number of distributions k. This is a novel statistical-computational trade-off for density estimation, demonstrating that any data structure must use close to a linear number of samples or take close to linear query time. The lower bound holds even in the realizable case where q=pi for some i, and when the distributions are flat (specifically, all distributions are uniform over half of the domain [n]). We also give a simple data structure for our lower bound instance with asymptotically matching upper bounds. Experiments show that the data structure is quite efficient in practice.