Low-degree learning and the metric entropy of polynomials

Let mathscrFn,d be the class of all functions f:-1,1n→[-1,1] on the n-dimensional discrete hypercube of degree at most d. In the first part of this paper, we prove that any (deterministic or randomized) algorithm which learns mathscrFn,d with L2-accuracy varepsilon requires at least Omega((1-√varepsilon)2dlog n) queries for large enough n, thus establishing the sharpness as n→∞ of a recent upper bound of Eskenazis and Ivanisvili (2021). To do this, we show that the L2-packing numbers mathsfM(mathscrFn,d,|·|L2,varepsilon) of the concept class mathscrFn,d satisfy the two-sided estimate c(1-varepsilon)2dlog n ≤ log mathsfM(mathscrFn,d,|·|L2,varepsilon) ≤ frac2Cdlog nvarepsilon4 for large enough n, where c, C>0 are universal constants. In the second part of the paper, we present a logarithmic upper bound for the randomized query complexity of classes of bounded approximate polynomials whose Fourier spectra are concentrated on few subsets. As an application, we prove new estimates for the number of random queries required to learn approximate juntas of a given degree, functions with rapidly decaying Fourier tails and constant depth circuits of given size. Finally, we obtain bounds for the number of queries required to learn the polynomial class mathscrFn,d without error in the query and random example models.