# Adaptive Huber Regression

arXiv: https://arxiv.org/abs/1706.06991 · HTML: https://mlsanity.com/paper/1706.06991/

## Key facts

- Cited by 60 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Jun 2017.

## Built on by (most cited first, 30 of 60)

- [How do noise tails impact on deep ReLU networks?](https://mlsanity.com/paper/2203.10418/) (2022)
- [Towards Robust Offline Reinforcement Learning under Diverse Data Corruption](https://mlsanity.com/paper/2310.12955/) (2023)
- [High-dimensional robust regression under heavy-tailed data: Asymptotics and Universality](https://mlsanity.com/paper/2309.16476/) (2023)
- [Data-Driven Sample Average Approximation with Covariate Information](https://mlsanity.com/paper/2207.13554/) (2022)
- [Tackling Heavy-Tailed Rewards in Reinforcement Learning with Function Approximation: Minimax Optimal and Instance-Dependent Regret Bounds](https://mlsanity.com/paper/2306.06836/) (2023)
- [High Dimensional Statistical Estimation under Uniformly Dithered One-bit Quantization](https://mlsanity.com/paper/2202.13157/) (2022)
- [Heavy-tailed Streaming Statistical Estimation](https://mlsanity.com/paper/2108.11483/) (2021)
- [Communication-Constrained Distributed Quantile Regression with Optimal Statistical Guarantees](https://mlsanity.com/paper/2110.13113/) (2021)
- [Variance-aware robust reinforcement learning with linear function approximation under heavy-tailed rewards](https://mlsanity.com/paper/2303.05606/) (2023)
- [Low-rank Matrix Bandits with Heavy-tailed Rewards](https://mlsanity.com/paper/2404.17709/) (2024)
- [Non-Asymptotic Guarantees for Robust Statistical Learning under Infinite Variance Assumption](https://mlsanity.com/paper/2201.03182/) (2022)
- [Higher degree sum-of-squares relaxations robust against oblivious outliers](https://mlsanity.com/paper/2211.07327/) (2022)
- [Heteroscedasticity-aware residuals-based contextual stochastic optimization](https://mlsanity.com/paper/2101.03139/) (2021)
- [Are Latent Factor Regression and Sparse Regression Adequate?](https://mlsanity.com/paper/2203.01219/) (2022)
- [Forecasting SQL Query Cost at Twitter](https://mlsanity.com/paper/2204.05529/) (2022)
- [Heavy-Tailed Linear Bandits: Huber Regression with One-Pass Update](https://mlsanity.com/paper/2503.00419/) (2025)
- [Robust Matrix Completion with Heavy-tailed Noise](https://mlsanity.com/paper/2206.04276/) (2022)
- [Outlier Robust and Sparse Estimation of Linear Regression Coefficients](https://mlsanity.com/paper/2208.11592/) (2022)
- [Computationally Efficient and Statistically Optimal Robust High-Dimensional Linear Regression](https://mlsanity.com/paper/2305.06199/) (2023)
- [Online Estimation and Inference for Robust Policy Evaluation in Reinforcement Learning](https://mlsanity.com/paper/2310.02581/) (2023)
- [Improved Analysis of Sparse Linear Regression in Local Differential Privacy Model](https://mlsanity.com/paper/2310.07367/) (2023)
- [Improved Regret Bounds for Linear Bandits with Heavy-Tailed Rewards](https://mlsanity.com/paper/2506.04775/) (2025)
- [A Survey Of Regression Algorithms And Connections With Deep Learning](https://mlsanity.com/paper/2104.12647/) (2021)
- [Non-asymptotic analysis and inference for an outlyingness induced winsorized mean](https://mlsanity.com/paper/2105.02337/) (2021)
- [Robust Variable Selection and Estimation Via Adaptive Elastic Net S-Estimators for Linear Regression](https://mlsanity.com/paper/2107.03325/) (2021)
- [Robust Linear Regression for General Feature Distribution](https://mlsanity.com/paper/2202.02080/) (2022)
- [Click prediction boosting via Bayesian hyperparameter optimization based ensemble learning pipelines](https://mlsanity.com/paper/2206.03592/) (2022)
- [Robust convex biclustering with a tuning-free method](https://mlsanity.com/paper/2212.03122/) (2022)
- [Outlier-robust Estimation of a Sparse Linear Model Using Invexity](https://mlsanity.com/paper/2306.12678/) (2023)
- [Differentially Private Sparse Linear Regression with Heavy-tailed Responses](https://mlsanity.com/paper/2506.06861/) (2025)

## Abstract (arXiv)

Big data can easily be contaminated by outliers or contain variables with heavy-tailed distributions, which makes many conventional methods inadequate. To address this challenge, we propose the adaptive Huber regression for robust estimation and inference. The key observation is that the robustification parameter should adapt to the sample size, dimension and moments for optimal tradeoff between bias and robustness. Our theoretical framework deals with heavy-tailed distributions with bounded (1+δ)-th moment for any δ > 0. We establish a sharp phase transition for robust estimation of regression parameters in both low and high dimensions: when δ ≥ 1, the estimator admits a sub-Gaussian-type deviation bound without sub-Gaussian assumptions on the data, while only a slower rate is available in the regime 0<δ< 1. Furthermore, this transition is smooth and optimal. In addition, we extend the methodology to allow both heavy-tailed predictors and observation noise. Simulation studies lend further support to the theory. In a genetic study of cancer cell lines that exhibit heavy-tailedness, the proposed methods are shown to be more robust and predictive.