# On the Robustness of Kernel Ridge Regression Using the Cauchy Loss Function

arXiv: https://arxiv.org/abs/2503.20120 · HTML: https://mlsanity.com/paper/2503.20120/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 45 references that are in the map.
- First posted on arXiv 26 Mar 2025.

## Builds on (most cited first, 8 of 8)

- [Fast rates in statistical and online learning](https://mlsanity.com/paper/1507.02592/) (2015)
- [Empirical risk minimization for heavy-tailed losses](https://mlsanity.com/paper/1406.2462/) (2014)
- [Optimal Algorithms for Stochastic Multi-Armed Bandits with Heavy Tailed Rewards](https://mlsanity.com/paper/2010.12866/) (2020)
- [Consistent regression when oblivious outliers overwhelm](https://mlsanity.com/paper/2009.14774/) (2020)
- [Learning with Non-Convex Truncated Losses by SGD](https://mlsanity.com/paper/1805.07880/) (2018)
- [Non-Asymptotic Guarantees for Robust Statistical Learning under Infinite Variance Assumption](https://mlsanity.com/paper/2201.03182/) (2022)
- [Cauchy Loss Function: Robustness Under Gaussian and Cauchy Noise](https://mlsanity.com/paper/2302.07238/) (2023)
- [Massive MIMO with Cauchy Noise: Channel Estimation, Achievable Rate and Data Decoding](https://mlsanity.com/paper/2307.02724/) (2023)

## Built on by (most cited first, 2 of 2)

- [Regularized least squares learning with heavy-tailed noise is minimax optimal](https://mlsanity.com/paper/2505.14214/) (2025)
- [Cubit: Token Mixer with Kernel Ridge Regression](https://mlsanity.com/paper/2605.06501/) (2026)

## Abstract (arXiv)

Robust regression aims to develop methods for estimating an unknown regression function in the presence of outliers, heavy-tailed distributions, or contaminated data, which can severely impact performance. Most existing theoretical results in robust regression assume that the noise has a finite absolute mean, an assumption violated by certain distributions, such as Cauchy and some Pareto noise. In this paper, we introduce a generalized Cauchy noise framework that accommodates all noise distributions with finite moments of any order, even when the absolute mean is infinite. Within this framework, we study the textitkernel Cauchy ridge regressor (textitKCRR), which minimizes a regularized empirical Cauchy risk to achieve robustness. To derive the L2-risk bound for KCRR, we establish a connection between the excess Cauchy risk and L2-risk for sufficiently large scale parameters of the Cauchy loss, which reveals that these two risks are equivalent. Furthermore, under the assumption that the regression function satisfies H"older smoothness, we derive excess Cauchy risk bounds for KCRR, showing improved performance as the scale parameter decreases. By considering the twofold effect of the scale parameter on the excess Cauchy risk and its equivalence with the L2-risk, we establish the almost minimax-optimal convergence rate for KCRR in terms of L2-risk, highlighting the robustness of the Cauchy loss in handling various types of noise. Finally, we validate the effectiveness of KCRR through experiments on both synthetic and real-world datasets under diverse noise corruption scenarios.