# Fast Rates for Nonstationary Weighted Risk Minimization

arXiv: https://arxiv.org/abs/2602.05742 · HTML: https://mlsanity.com/paper/2602.05742/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 36 references that are in the map.
- First posted on arXiv 5 Feb 2026.

## Builds on (most cited first, 10 of 10)

- [Nonparametric regression using deep neural networks with ReLU activation function](https://mlsanity.com/paper/1708.06633/) (2017)
- [An Adaptive Algorithm for Learning with Unknown Distribution Drift](https://mlsanity.com/paper/2305.02252/) (2023)
- [Statistical Learning under Nonstationary Mixing Processes](https://mlsanity.com/paper/1512.08064/) (2015)
- [Covering Numbers for Deep ReLU Networks with Applications to Function Approximation and Nonparametric Regression](https://mlsanity.com/paper/2410.06378/) (2024)
- [A Stability Principle for Learning under Non-Stationarity](https://mlsanity.com/paper/2310.18304/) (2023)
- [Weighted Empirical Risk Minimization: Sample Selection Bias Correction based on Importance Sampling](https://mlsanity.com/paper/2002.05145/) (2020)
- [Generalization bounds for nonparametric regression with β-mixing samples](https://mlsanity.com/paper/2108.00997/) (2021)
- [Three Quantization Regimes for ReLU Networks](https://mlsanity.com/paper/2405.01952/) (2024)
- [Trade-off Between Dependence and Complexity for Nonparametric Learning -- an Empirical Process Approach](https://mlsanity.com/paper/2401.08978/) (2024)
- [Optimal Empirical Risk Minimization under Temporal Distribution Shifts](https://mlsanity.com/paper/2507.13287/) (2025)

## Built on by (most cited first, 1 of 1)

- [Beyond Discrepancy: Oracle Drift in Nonstationary Weighted Risk Minimization](https://mlsanity.com/paper/2610.13598/) (2026)

## Abstract (arXiv)

Weighted empirical risk minimization is a common approach to prediction under distribution drift. This article studies its out-of-sample prediction error under nonstationarity. We provide a general decomposition of the excess risk into a learning term and an error term associated with distribution drift, and prove oracle inequalities for the learning error under mixing conditions. The learning bound holds uniformly over arbitrary weight classes and accounts for the effective sample size induced by the weight vector, the complexity of the weight and hypothesis classes, and potential data dependence. We also analyze the drift error in the square loss setting and provide explicit bounds for structural-break and smoothly drifting processes. We then combine the learning and drift analysis in (auto-) regression problems with linear models, basis approximations, and neural networks. In all cases, the optimal weights are characterized by a bias--variance trade-off between learning and drift, while minimax-optimal rates (up to logarithmic factors) are recovered when specialized to unweighted and stationary settings.