mlsanityData to 7 Oct 2026

Implicit Bias of Mirror Flow on Separable Data

arXiv 2406.12763 ↗First posted 18 Jun 2024stat.ML cs.LG math.OCcited by 7

Not on the map. Why?

Abstract (arXiv)

We examine the continuous-time counterpart of mirror descent, namely mirror flow, on classification problems which are linearly separable. Such problems are minimised `at infinity' and have many possible solutions; we study which solution is preferred by the algorithm depending on the mirror potential. For exponential tailed losses and under mild assumptions on the potential, we show that the iterates converge in direction towards a φ_∞-maximum margin classifier. The function φ_∞ is the textithorizon function of the mirror potential and characterises its shape `at infinity'. When the potential is separable, a simple formula allows to compute this function. We analyse several examples of potentials and provide numerical experiments highlighting our results.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 26 of its 50 references that are in the map. First posted on arXiv 18 Jun 2024.

Builds on

26 of 50 references.

References · 26

  1. 2016Understanding deep learning requires rethinking generalizationcited by 1,657
  2. 2017The Implicit Bias of Gradient Descent on Separable Datacited by 649
  3. 2017Implicit Regularization in Matrix Factorizationcited by 318
  4. 2019Gradient Descent Maximizes the Margin of Homogeneous Neural Networkscited by 308
  5. 2020Kernel and Rich Regimes in Overparametrized Modelscited by 302
  6. 2020Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Losscited by 267
  7. 2018Characterizing Implicit Bias in Terms of Optimization Geometrycited by 259
  8. 2018Convergence of Gradient Descent on Separable Datacited by 108
  9. 2021Implicit Bias of SGD for Diagonal Linear Networks: a Provable Benefit of Stochasticitycited by 107
  10. 2018Risk and parameter convergence of logistic regressioncited by 81
  11. 2019Implicit Regularization for Optimal Sparse Recoverycited by 78
  12. 2020A Unifying View on Implicit Bias in Training Linear Neural Networkscited by 74
14 more
  1. 2020Implicit Bias in Deep Linear Classification: Initialization Scale vs Training Accuracycited by 70
  2. 2021On the Implicit Bias of Initialization Shape: Beyond Infinitesimal Mirror Descentcited by 69
  3. 2018Stochastic Gradient Descent on Separable Data: Exact Convergence with a Fixed Learning Ratecited by 63
  4. 2020Gradient descent follows the regularization path for general lossescited by 57
  5. 2023Saddle-to-Saddle Dynamics in Diagonal Linear Networkscited by 54
  6. 2019Lexicographic and Depth-Sensitive Margins in Homogeneous and Non-Homogeneous Deep Modelscited by 53
  7. 2019Bad Global Minima Exist and SGD Can Reach Themcited by 49
  8. 2022Implicit Bias of Gradient Descent on Reparametrized Models: On Equivalence to Mirror Descentcited by 38
  9. 2023(S)GD over Diagonal Linear Networks: Implicit Regularisation, Large Stepsizes and Edge of Stabilitycited by 35
  10. 2022Mirror Descent Maximizes Generalized Margin and Can Be Implemented Efficientlycited by 25
  11. 2023A Unified Approach to Controlling Implicit Regularization via Mirror Descentcited by 13
  12. 2021Implicit Regularization in Matrix Sensing via Mirror Descentcited by 11
  13. 2024Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networkscited by 9
  14. 2022Astral Space: Convex Analysis at Infinitycited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 5.

7 papers.

  1. 2025Mirror, Mirror of the Flow: How Does Regularization Shape Implicit Bias?cited by 10
  2. 2025Diagonalizing the Softmax: Hadamard Initialization for Tractable Cross-Entropy Dynamicscited by 2
  3. 2026Never Saddle for Reparameterized Steepest Descent as Mirror Flowcited by 2
  4. 2026Hard labels sampled from sparse targets mislead rotation invariant algorithmscited by 0
  5. 2026Implicit Bias of Mirror Flow in Homogeneous Neural Networks: Sparse and Dense Feature Learningcited by 0
  6. 2026HORST: Composing Optimizer Geometries for Sparse Transformer Trainingcited by 0
  7. 2026The Implicit Bias of Depth: From Neural Collapse to Softmax Codescited by 0