mlsanityData to 7 Oct 2026

On the implicit regularization of Langevin dynamics with projected noise

arXiv 2602.12257 ↗First posted 12 Feb 2026math.PR cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

We study Langevin dynamics with noise projected onto the directions orthogonal to an isometric group action. This mathematical model is introduced to shed new light on the effects of symmetry on stochastic gradient descent for over-parametrized models. Our main result identifies a novel form of implicit regularization: when the initial and target density are both invariant under the group action, Langevin dynamics with projected noise is equivalent in law to Langevin dynamics with isotropic diffusion but with an additional drift term proportional to the negative log volume of the group orbit. We prove this result by constructing a coupling of the two processes via a third process on the group itself, and identify the additional drift as the mean curvature of the orbits.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 16 of its 41 references that are in the map. First posted on arXiv 12 Feb 2026.

Builds on

16 of 41 references.

References · 16

  1. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,337
  2. 2017The Implicit Bias of Gradient Descent on Separable Datacited by 649
  3. 2019Implicit Regularization in Deep Matrix Factorizationcited by 402
  4. 2014In Search of the Real Inductive Bias: On the Role of Implicit Regularization in Deep Learningcited by 339
  5. 2017Implicit Regularization in Matrix Factorizationcited by 318
  6. 2019Gradient Descent Maximizes the Margin of Homogeneous Neural Networkscited by 308
  7. 2018On the Optimization of Deep Networks: Implicit Acceleration by Overparameterizationcited by 280
  8. 2017Non-convex learning via Stochastic Gradient Langevin Dynamics: a nonasymptotic analysiscited by 242
  9. 2018Gradient descent aligns the layers of deep linear networkscited by 182
  10. 2021Implicit Bias of SGD for Diagonal Linear Networks: a Provable Benefit of Stochasticitycited by 107
  11. 2020The Implicit Regularization of Stochastic Gradient Flow for Least Squarescited by 58
  12. 2023(S)GD over Diagonal Linear Networks: Implicit Regularisation, Large Stepsizes and Edge of Stabilitycited by 35
4 more
  1. 2024The geometry of the deep linear networkcited by 10
  2. 2025Implicit Bias of Gradient Descent for Non-Homogeneous Deep Networkscited by 9
  3. 2025An entropy formula for the Deep Linear Networkcited by 8
  4. 2023Riemannian Langevin Monte Carlo schemes for sampling PSD matrices with fixed rankcited by 1

Built on by

Not cited yet.