mlsanityData to 14 Oct 2026

Ridge Regression from Poisson Resetting: A Renewal Perspective on Spectral Regularization

arXiv 2605.30059 ↗First posted 28 May 2026cs.LG cond-mat.stat-mech stat.MLcited by 0

Not on the map. Why?

Abstract (arXiv)

We connect stochastic resetting from non-equilibrium statistical physics with ridge regularization in statistical learning. For linear gradient flow, resetting to the origin at rate r produces stationary mean (X^top X+rI)-1X^top y, exactly the ridge estimator with penalty λ=r. This uses the known Laplace-transform relationship between ridge regression and exponential-time averaging of gradient flow, with the exponential time now interpreted as the stationary age associated with Poisson resetting. We then extend this identity to general renewal reset laws: the exponential reset time distribution is the unique renewal law whose stationary mean reproduces scalar ridge in every eigendirection as an exact filter identity for every positive curvature, while non-exponential renewal laws generate alternative spectral filters. At the fluctuation level, we study a separate additive Ornstein-Uhlenbeck extension with constant diffusion, interpreted as a stylized SGD approximation. In this setting, the equality holds only at the level of the mean, since the reset process has a nonzero stationary covariance from accumulated OU noise and reset-timing variance, whereas deterministic ridge is a fixed estimator with the same center. Stylized experiments compare the deterministic renewal-induced filters directly and illustrate when filters induced by non-exponential reset-time laws can differ predictively from ridge. The results for the stationary mean and the induced spectral filters are established for continuous-time gradient flow with isotropic resetting on quadratic objectives; the covariance and risk formulas additionally assume additive noise with state-independent covariance.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 16 of its 46 references that are in the map. First posted on arXiv 28 May 2026.

Builds on

16 of 46 references.

References · 16

  1. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  2. 2017Stochastic Gradient Descent as Approximate Bayesian Inferencecited by 300
  3. 2022The Primacy Bias in Deep Reinforcement Learningcited by 247
  4. 2015A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insightscited by 206
  5. 2020Implicit Gradient Regularizationcited by 146
  6. 2019Implicit regularization for deep neural networks driven by an Ornstein-Uhlenbeck like processcited by 123
  7. 2018A Continuous-Time View of Early Stopping for Least Squarescited by 69
  8. 2020The Implicit Regularization of Stochastic Gradient Flow for Least Squarescited by 59
  9. 2021The Benefits of Implicit Regularization from SGD in Least Squares Problemscited by 32
  10. 2022Implicit Regularization or Implicit Conditioning? Exact Risk Trajectories of SGD in High Dimensionscited by 23
  11. 2024Non-Stationary Learning of Neural Networks with Automatic Soft Parameter Resetcited by 17
  12. 2024On Regularization via Early Stopping for Least Squares Regressioncited by 11
4 more
  1. 2021Comparing Classes of Estimators: When does Gradient Descent Beat Ridge Regression in Linear Models?cited by 5
  2. 2024Stochastic Resetting Mitigates Latent Gradient Bias of SGD from Label Noisecited by 4
  3. 2025First-Passage Approach to Optimizing Perturbations for Improved Training of Machine Learning Modelscited by 3
  4. 2025On the Saturation Effects of Spectral Algorithms in Large Dimensionscited by 2

Built on by

Not cited yet.