# Generalized Kernel Regularized Least Squares

arXiv: https://arxiv.org/abs/2209.14355 · HTML: https://mlsanity.com/paper/2209.14355/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 47 references that are in the map.
- First posted on arXiv 28 Sep 2022.

## Builds on (most cited first, 5 of 5)

- [Meta-learners for Estimating Heterogeneous Treatment Effects using Machine Learning](https://mlsanity.com/paper/1706.03461/) (2017)
- [Quasi-Oracle Estimation of Heterogeneous Treatment Effects](https://mlsanity.com/paper/1712.04912/) (2017)
- [Randomized sketches for kernels: Fast and optimal non-parametric regression](https://mlsanity.com/paper/1501.06195/) (2015)
- [Smoothing parameter and model selection for general smooth models](https://mlsanity.com/paper/1511.03864/) (2015)
- [DoubleML -- An Object-Oriented Implementation of Double Machine Learning in R](https://mlsanity.com/paper/2103.09603/) (2021)

## Built on by (most cited first, 1 of 1)

- [Inference at the data's edge: Gaussian processes for modeling and inference under model-dependency, poor overlap, and extrapolation](https://mlsanity.com/paper/2407.10442/) (2024)

## Abstract (arXiv)

Kernel Regularized Least Squares (KRLS) is a popular method for flexibly estimating models that may have complex relationships between variables. However, its usefulness to many researchers is limited for two reasons. First, existing approaches are inflexible and do not allow KRLS to be combined with theoretically-motivated extensions such as random effects, unregularized fixed effects, or non-Gaussian outcomes. Second, estimation is extremely computationally intensive for even modestly sized datasets. Our paper addresses both concerns by introducing generalized KRLS (gKRLS). We note that KRLS can be re-formulated as a hierarchical model thereby allowing easy inference and modular model construction where KRLS can be used alongside random effects, splines, and unregularized fixed effects. Computationally, we also implement random sketching to dramatically accelerate estimation while incurring a limited penalty in estimation quality. We demonstrate that gKRLS can be fit on datasets with tens of thousands of observations in under one minute. Further, state-of-the-art techniques that require fitting the model over a dozen times (e.g. meta-learners) can be estimated quickly.