# Small noise analysis for Tikhonov and RKHS regularizations

arXiv: https://arxiv.org/abs/2305.11055 · HTML: https://mlsanity.com/paper/2305.11055/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 40 references that are in the map.
- First posted on arXiv 18 May 2023.

## Builds on (most cited first, 5 of 5)

- [Learning interaction kernels in mean-field equations of 1st-order systems of interacting particles](https://mlsanity.com/paper/2010.15694/) (2020)
- [A data-driven peridynamic continuum model for upscaling molecular dynamics](https://mlsanity.com/paper/2108.04883/) (2021)
- [Nonparametric learning of kernels in nonlocal operators](https://mlsanity.com/paper/2205.11006/) (2022)
- [Data adaptive RKHS Tikhonov regularization for learning kernels in operators](https://mlsanity.com/paper/2203.03791/) (2022)
- [A Data-Adaptive Prior for Bayesian Learning of Kernels in Operators](https://mlsanity.com/paper/2212.14163/) (2022)

## Abstract (arXiv)

Regularization plays a pivotal role in ill-posed machine learning and inverse problems. However, the fundamental comparative analysis of various regularization norms remains open. We establish a small noise analysis framework to assess the effects of norms in Tikhonov and RKHS regularizations, in the context of ill-posed linear inverse problems with Gaussian noise. This framework studies the convergence rates of regularized estimators in the small noise limit and reveals the potential instability of the conventional L2-regularizer. We solve such instability by proposing an innovative class of adaptive fractional RKHS regularizers, which covers the L2 Tikhonov and RKHS regularizations by adjusting the fractional smoothness parameter. A surprising insight is that over-smoothing via these fractional RKHSs consistently yields optimal convergence rates, but the optimal hyper-parameter may decay too fast to be selected in practice.