Near-Optimal Learning of Tree-Structured Distributions by Chow-Liu

We provide finite sample guarantees for the classical Chow-Liu algorithm (IEEE Trans.~Inform.~Theory, 1968) to learn a tree-structured graphical model of a distribution. For a distribution P on Sigman and a tree T on n nodes, we say T is an varepsilon-approximate tree for P if there is a T-structured distribution Q such that D(P;||;Q) is at most varepsilon more than the best possible tree-structured distribution for P. We show that if P itself is tree-structured, then the Chow-Liu algorithm with the plug-in estimator for mutual information with widetildeO(|Sigma|3 nvarepsilon-1) i.i.d.~samples outputs an varepsilon-approximate tree for P with constant probability. In contrast, for a general P (which may not be tree-structured), Omega(n2varepsilon-2) samples are necessary to find an varepsilon-approximate tree. Our upper bound is based on a new conditional independence tester that addresses an open problem posed by Canonne, Diakonikolas, Kane, and Stewart~(STOC, 2018): we prove that for three random variables X,Y,Z each over Sigma, testing if I(X; Y mid Z) is 0 or ≥ varepsilon is possible with widetildeO(|Sigma|3/varepsilon) samples. Finally, we show that for a specific tree T, with widetildeO (|Sigma|2nvarepsilon-1) samples from a distribution P over Sigman, one can efficiently learn the closest T-structured distribution in KL divergence by applying the add-1 estimator at each node.