mlsanityData to 7 Oct 2026

On Empirical Risk Minimization with Dependent and Heavy-Tailed Data

arXiv 2109.02224 ↗First posted 6 Sep 2021math.ST stat.ML stat.THcited by 16

Not on the map. Why?

Abstract (arXiv)

In this work, we establish risk bounds for the Empirical Risk Minimization (ERM) with both dependent and heavy-tailed data-generating processes. We do so by extending the seminal works of Mendelson [Men15, Men18] on the analysis of ERM with heavy-tailed but independent and identically distributed observations, to the strictly stationary exponentially β-mixing case. Our analysis is based on explicitly controlling the multiplier process arising from the interaction between the noise and the function evaluations on inputs. It allows for the interaction to be even polynomially heavy-tailed, which covers a significantly large class of heavy-tailed models beyond what is analyzed in the learning theory literature. We illustrate our results by deriving rates of convergence for the high-dimensional linear regression problem with dependent and heavy-tailed data.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 17 of its 72 references that are in the map. First posted on arXiv 6 Sep 2021.

Builds on

17 of 72 references.

References · 17

  1. 2014Learning without Concentrationcited by 50
  2. 2015Learning with Square Loss: Localization through Offset Rademacher Complexitycited by 37
  3. 2015Statistical consistency and asymptotic normality for high-dimensional robust M-estimatorscited by 27
  4. 2016Fast Rates for General Unbounded Loss Functions: from ERM to Generalized Bayescited by 27
  5. 2017Learning Whenever Learning is Possible: Universal Learning under General Stochastic Processescited by 23
  6. 2020Statistical Learning with Conditional Value at Riskcited by 19
  7. 2014A Kernel Independence Test for Random Processescited by 18
  8. 2020Sharp Concentration Results for Heavy-Tailed Distributionscited by 15
  9. 2014Learning without Concentration for General Loss Functionscited by 13
  10. 2019Learning from weakly dependent data under Dobrushin's conditioncited by 13
  11. 2019High-dimensional vector autoregressive time series modeling via tensor decompositioncited by 13
  12. 2020PAC-Bayesian Bound for the Conditional Value at Riskcited by 13
5 more
  1. 2017Regularization, sparse recovery, and median-of-means tournamentscited by 11
  2. 2019Excess risk bounds in robust empirical risk minimizationcited by 9
  3. 2021Statistical Estimation from Dependent Datacited by 9
  4. 2020On Learnability under General Stochastic Processescited by 4
  5. 2021On Monte-Carlo methods in convex stochastic optimizationcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 6; 2024: 6; 2025: 1; 2026: 1.

16 papers.

Sort:

By area:

  1. 2022Learning with little mixingcited by 33
  2. 2023Towards Robust Offline Reinforcement Learning under Diverse Data Corruptioncited by 24
  3. 2023High-dimensional robust regression under heavy-tailed data: Asymptotics and Universalitycited by 15
  4. 2024Sharp Rates in Dependent Learning Theory: Avoiding Sample Size Deflation for the Square Losscited by 10
  5. 2024Uncertainty-based Offline Variational Bayesian Reinforcement Learning for Robustness under Diverse Data Corruptionscited by 8
  6. 2022Constrained Stochastic Nonconvex Optimization with State-dependent Markov Datacited by 6
  7. 2023The noise level in linear regression with dependent datacited by 6
  8. 2025Dual-Robust Cross-Domain Offline Reinforcement Learning Against Dynamics Shiftscited by 5
  9. 2023PAC-Bayes Generalisation Bounds for Dynamical Systems Including Stable RNNscited by 4
  10. 2024Trade-off Between Dependence and Complexity for Nonparametric Learning -- an Empirical Process Approachcited by 2
  11. 2023Neural Network Approximation for Pessimistic Offline Reinforcement Learningcited by 1
  12. 2024Rate-Optimal Non-Asymptotics for the Quadratic Prediction Error Methodcited by 1
4 more
  1. 2023Empirical Risk Minimization for Losses without Variancecited by 0
  2. 2024Subsampled Ensemble Can Improve Generalization Tail Exponentiallycited by 0
  3. 2024Data Quality in Edge Machine Learning: A State-of-the-Art Surveycited by 0
  4. 2026TailedTS: Benchmark Dataset for Heavy-Tailed Time Series Prediction and Periodicity Quantificationcited by 0