Adaptivity for Regularized Kernel Methods by Lepskii's Principle

We address the problem of it adaptivity in the framework of reproducing kernel Hilbert space (RKHS) regression. More precisely, we analyze estimators arising from a linear regularization scheme g_lam. In practical applications, an important task is to choose the regularization parameter lam appropriately, i.e. based only on the given data and independently on unknown structural assumptions on the regression function. An attractive approach avoiding data-splitting is the it Lepskii Principle (LP), also known as the it Balancing Principle is this setting. We show that a modified parameter choice based on (LP) is minimax optimal adaptive, up to loglog(n). A convenient result is the fact that balancing in L2(nu)- norm, which is easiest, automatically gives optimal balancing in all stronger norms, interpolating between L2(nu) and the RKHS. An analogous result is open for other classical approaches to data dependent choices of the regularization parameter, e.g. for Hold-Out.