Nonparametric approximation of conditional expectation operators

Given the joint distribution of two random variables X,Y on some second countable locally compact Hausdorff space, we investigate the statistical approximation of the L2-operator defined by [Pf](x) := mathbbE[ f(Y) mid X = x ] under minimal assumptions. By modifying its domain, we prove that P can be arbitrarily well approximated in operator norm by Hilbert-Schmidt operators acting on a reproducing kernel Hilbert space. This fact allows to estimate P uniformly by finite-rank operators over a dense subspace even when P is not compact. In terms of modes of convergence, we thereby obtain the superiority of kernel-based techniques over classically used parametric projection approaches such as Galerkin methods. This also provides a novel perspective on which limiting object the nonparametric estimate of P converges to. As an application, we show that these results are particularly important for a large family of spectral analysis techniques for Markov transition operators. Our investigation also gives a new asymptotic perspective on the so-called kernel conditional mean embedding, which is the theoretical foundation of a wide variety of techniques in kernel-based nonparametric inference.