mlsanityData to 7 Oct 2026

Mean estimation and regression under heavy-tailed distributions--a survey

arXiv 1906.04280 ↗First posted 10 Jun 2019math.ST cs.LG stat.MLcited by 133

Not on the map. Why?

Abstract (arXiv)

We survey some of the recent advances in mean estimation and regression function estimation. In particular, we describe sub-Gaussian mean estimators for possibly heavy-tailed data both in the univariate and multivariate settings. We focus on estimators based on median-of-means techniques but other methods such as the trimmed mean and Catoni's estimator are also reviewed. We give detailed proofs for the cornerstone results. We dedicate a section on statistical learning problems--in particular, regression function estimation--in the presence of possibly heavy-tailed data.

Key facts. Cited by 133 papers in the map (data through 7 Oct 2026). First posted on arXiv 10 Jun 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 19; 2022: 27; 2023: 22; 2024: 15; 2025: 23; 2026: 27.

133 papers.

Sort:

By area:

  1. 2021High-probability Bounds for Non-Convex Stochastic Optimization with Heavy Tailscited by 80
  2. 2021High-Dimensional Experimental Design and Kernel Banditscited by 55
  3. 2021Efficient Mean Estimation with Pure Differential Privacy via a Sum-of-Squares Exponential Mechanismcited by 50
  4. 2021First-Order Regret in Reinforcement Learning with Linear Function Approximation: A Robust Estimation Approachcited by 38
  5. 2021Regret Minimization in Heavy-Tailed Banditscited by 24
  6. 2022Mirror Descent Strikes Again: Optimal Stochastic Convex Optimization under Infinite Noise Variancecited by 22
  7. 2024The Utility and Complexity of in- and out-of-Distribution Machine Unlearningcited by 20
  8. 2021Distribution-Free Robust Linear Regressioncited by 19
  9. 2021Nearly Optimal Algorithms for Level Set Estimationcited by 17
  10. 2022PopArt: Efficient Sparse Regression and Experimental Design for Optimal Sparse Linear Banditscited by 17
  11. 2021Near-Optimal Quantum Algorithms for Multivariate Mean Estimationcited by 16
  12. 2022Instance-optimal PAC Algorithms for Contextual Banditscited by 16
38 more
  1. 2024Heavy-Tailed Diffusion with Denoising L'evy Probabilistic Modelscited by 16
  2. 2021Quantum Sub-Gaussian Mean Estimatorcited by 15
  3. 2022Learning Mixtures of Linear Dynamical Systemscited by 15
  4. 2022Quantum policy gradient algorithmscited by 15
  5. 2023High-dimensional robust regression under heavy-tailed data: Asymptotics and Universalitycited by 15
  6. 2021Collaborative Pure Exploration in Kernel Banditcited by 13
  7. 2022Outlier-Robust Sparse Mean Estimation for Heavy-Tailed Distributionscited by 13
  8. 2023Tackling Heavy-Tailed Rewards in Reinforcement Learning with Function Approximation: Minimax Optimal and Instance-Dependent Regret Boundscited by 13
  9. 2022Algorithmic Stability of Heavy-Tailed Stochastic Gradient Descent on Least Squarescited by 12
  10. 2022A Non-Asymptotic Moreau Envelope Theory for High-Dimensional Generalized Linear Modelscited by 11
  11. 2024High-probability minimax lower boundscited by 11
  12. 2021Heavy-tailed Streaming Statistical Estimationcited by 10
  13. 2022Minimax risk classifiers with 0-1 losscited by 9
  14. 2022Exponential Tail Local Rademacher Complexity Risk Bounds Without the Bernstein Conditioncited by 9
  15. 2022Mitigating multiple descents: A model-agnostic framework for risk monotonizationcited by 9
  16. 2023Provably Robust Temporal Difference Learning for Heavy-Tailed Rewardscited by 9
  17. 2021On the benefits of maximum likelihood estimation for Regression and Forecastingcited by 8
  18. 2023Convergence and concentration properties of constant step-size SGD through Markov chainscited by 8
  19. 2024Treatment of Statistical Estimation Problems in Randomized Smoothing for Adversarial Robustnesscited by 8
  20. 2022Quantizing Heavy-tailed Data in Statistical Estimation: (Near) Minimax Rates, Covariate Quantization, and Uniform Recoverycited by 7
  21. 2023Regret Distribution in Stochastic Bandits: Optimal Trade-off between Expectation and Tail Riskcited by 7
  22. 2023Efficient median of means estimatorcited by 7
  23. 2023Robust Offline Reinforcement learning with Heavy-Tailed Rewardscited by 7
  24. 2021Selective Sampling for Online Best-arm Identificationcited by 6
  25. 2022Non-Asymptotic Guarantees for Robust Statistical Learning under Infinite Variance Assumptioncited by 6
  26. 2022Many processors, little time: MCMC for partitions via optimal transport couplingscited by 6
  27. 2023On Private and Robust Banditscited by 6
  28. 2023Time-Uniform Confidence Spheres for Means of Random Vectorscited by 6
  29. 2024Dimension-free Private Mean Estimation for Anisotropic Distributionscited by 6
  30. 2025Beyond Statistical Learning: Exact Learning Is Essential for General Intelligencecited by 6
  31. 2022Robust Estimation of Discrete Distributions under Local Differential Privacycited by 5
  32. 2022Robust Methods for High-Dimensional Linear Learningcited by 5
  33. 2022Robust Online and Distributed Mean Estimation Under Adversarial Data Corruptioncited by 5
  34. 2025Heavy-Tailed Linear Bandits: Huber Regression with One-Pass Updatecited by 5
  35. 2026Order-Optimal Sequential 1-Bit Mean Estimation in General Tail Regimescited by 5
  36. 2021Robust Online Convex Optimization in the Presence of Outlierscited by 4
  37. 2021A Survey of Learning Criteria Going Beyond the Usual Riskcited by 4
  38. 2022Sample-optimal classical shadows for pure statescited by 4

50 most cited of 133 All 133 (JSON)