mlsanityData to 7 Oct 2026

Robust Estimation via Robust Gradient Estimation

arXiv 1802.06485 ↗First posted 19 Feb 2018stat.ML cs.AI cs.LGcited by 76

Not on the map. Why?

Abstract (arXiv)

We provide a new computationally-efficient class of estimators for risk minimization. We show that these estimators are robust for general statistical models: in the classical Huber epsilon-contamination model and in heavy-tailed settings. Our workhorse is a novel robust variant of gradient descent, and we provide conditions under which our gradient descent variant provides accurate estimators in a general convex risk minimization problem. We provide specific consequences of our theory for linear regression, logistic regression and for estimation of the canonical parameters in an exponential family. These results provide some of the first computationally tractable and provably robust estimators for these canonical statistical models. Finally, we study the empirical performance of our proposed methods on synthetic and real datasets, and find that our methods convincingly outperform a variety of baselines.

Key facts. Cited by 76 papers in the map (data through 7 Oct 2026). First posted on arXiv 19 Feb 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 21; 2022: 20; 2023: 13; 2024: 9; 2025: 4; 2026: 9.

76 papers.

Sort:

By area:

  1. 2022Policy Gradient Method For Robust Reinforcement Learningcited by 67
  2. 2021Robust and Differentially Private Mean Estimationcited by 66
  3. 2021Differential privacy and robust statistics in high dimensionscited by 64
  4. 2022Byzantine Machine Learning Made Easy by Resilient Averaging of Momentumscited by 63
  5. 2021Property Inference From Poisoningcited by 56
  6. 2021DORO: Distributional and Outlier Robust Optimizationcited by 54
  7. 2021High Dimensional Differentially Private Stochastic Optimization with Heavy-tailed Datacited by 42
  8. 2021On Proximal Policy Optimization's Heavy-tailed Gradientscited by 30
  9. 2022Robust Sparse Mean Estimation via Sum of Squarescited by 23
  10. 2023A New Approach to Learning Linear Dynamical Systemscited by 23
  11. 2021Clustering Mixture Models in Almost-Linear Time via List-Decodable Mean Estimationcited by 21
  12. 2022Streaming Algorithms for High-Dimensional Robust Statisticscited by 21
38 more
  1. 2021Robust Regression Revisited: Acceleration and Improved Estimation Ratescited by 15
  2. 2021Outlier-Robust Learning of Ising Models Under Dobrushin's Conditioncited by 14
  3. 2022Robust Constrained Reinforcement Learningcited by 14
  4. 2022Byzantine-Robust Online and Offline Distributed Reinforcement Learningcited by 13
  5. 2022Outlier-Robust Sparse Mean Estimation for Heavy-Tailed Distributionscited by 13
  6. 2022Estimation Contracts for Outlier-Robust Geometric Perceptioncited by 11
  7. 2023Robust Nonparametric Regression under Poisoning Attackcited by 11
  8. 2021Heavy-tailed Streaming Statistical Estimationcited by 10
  9. 2022AdaTerm: Adaptive T-Distribution Estimated Robust Moments for Noise-Robust Stochastic Gradient Optimizationcited by 10
  10. 2022Robust Imitation Learning from Corrupted Demonstrationscited by 10
  11. 2023Nearly-Linear Time and Streaming Algorithms for Outlier-Robust PCAcited by 10
  12. 2024SoS Certifiability of Subgaussian Distributions and its Algorithmic Applicationscited by 10
  13. 2023Near Optimal Private and Robust Linear Regressioncited by 9
  14. 2021On the benefits of maximum likelihood estimation for Regression and Forecastingcited by 8
  15. 2021Robust Estimation for Random Graphscited by 8
  16. 2025A Unified Theoretical Analysis of Private and Robust Offline Alignment: from RLHF to DPOcited by 8
  17. 2021Byzantine Fault-Tolerance in Federated Local SGD under 2f-Redundancycited by 7
  18. 2022Trimmed Maximum Likelihood Estimation for Robust Learning in Generalized Linear Modelscited by 6
  19. 2022Robust and Sparse Estimation of Linear Regression Coefficients with Heavy-tailed Noises and Covariatescited by 6
  20. 2023Time-Uniform Confidence Spheres for Means of Random Vectorscited by 6
  21. 2024Global Convergence Guarantees for Federated Policy Gradient Methods with Adversariescited by 6
  22. 2025On Corruption-Robustness in Performative Reinforcement Learningcited by 6
  23. 2022Robust Methods for High-Dimensional Linear Learningcited by 5
  24. 2022Efficient List-Decodable Regression using Batchescited by 5
  25. 2023Private Federated Learning with Autotuned Compressioncited by 5
  26. 2024Efficient PAC Learning of Halfspaces with Constant Malicious Noise Ratecited by 5
  27. 2021From inexact optimization to learning via gradient concentrationcited by 4
  28. 2021Robust Online Convex Optimization in the Presence of Outlierscited by 4
  29. 2022Robust estimation algorithms don't need to know the corruption levelcited by 4
  30. 2021Learning Deep Neural Networks under Agnostic Corrupted Supervisioncited by 3
  31. 2022Robust supervised learning with coordinate gradient descentcited by 3
  32. 2023Linear Regression using Heterogeneous Data Batchescited by 3
  33. 2023Robust Stochastic Optimization via Gradient Quantile Clippingcited by 3
  34. 2023Online Robust Mean Estimationcited by 3
  35. 2022On the Robustness of Random Forest Against Untargeted Data Poisoning: An Ensemble-Based Approachcited by 2
  36. 2024Robust Second-Order Nonconvex Optimization and Its Application to Low Rank Matrix Sensingcited by 2
  37. 2024On the Relevance of Byzantine Robust Optimization Against Data Poisoningcited by 2
  38. 2024Robust Gradient Descent for Phase Retrievalcited by 2

50 most cited of 76 All 76 (JSON)