# Robust Regression over Averaged Uncertainty

arXiv: https://arxiv.org/abs/2311.06960 · HTML: https://mlsanity.com/paper/2311.06960/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 47 references that are in the map.
- First posted on arXiv 12 Nov 2023.

## Builds on (most cited first, 9 of 9)

- [High-Dimensional Asymptotics of Prediction: Ridge Regression and Classification](https://mlsanity.com/paper/1507.03003/) (2015)
- [Distributionally Robust Logistic Regression](https://mlsanity.com/paper/1509.09259/) (2015)
- [Data-Driven Robust Optimization](https://mlsanity.com/paper/1401.0212/) (2013)
- [Regularization via Mass Transportation](https://mlsanity.com/paper/1710.10016/) (2017)
- [Optimal ridge penalty for real-world high-dimensional data can be zero or negative due to the implicit ridge regularization](https://mlsanity.com/paper/1805.10939/) (2018)
- [Oracle-Based Robust Optimization via Online Learning](https://mlsanity.com/paper/1402.6361/) (2014)
- [Tikhonov Regularization is Optimal Transport Robust under Martingale Constraints](https://mlsanity.com/paper/2210.01413/) (2022)
- [Wasserstein Logistic Regression with Mixed Features](https://mlsanity.com/paper/2205.13501/) (2022)
- [On the Complexity of the Weighted Fused Lasso](https://mlsanity.com/paper/1801.04987/) (2018)

## Abstract (arXiv)

We propose a new formulation of robust regression by integrating all realizations of the uncertainty set and taking an averaged approach to obtain the optimal solution for the ordinary least squares regression problem. We show that this formulation recovers ridge regression exactly and establishes the missing link between robust optimization and the mean squared error approaches for existing regression problems. We further demonstrate that the condition of this equivalence relies on the geometric properties of the defined uncertainty set. We provide exact, closed-form, in some cases, analytical solutions to the equivalent regularization strength under uncertainty sets induced by ℓp norm, Schatten p-norm, and general polytopes. We then show in synthetic datasets with different levels of uncertainties, a consistent improvement of the averaged formulation over the existing worst-case formulation in out-of-sample performance. In real-world regression problems obtained from UCI datasets, similar improvements are seen in the out-of-sample datasets.