Incremental Learning in Mirror Flows
Not on the map. Why?
Abstract (arXiv)
We study mirror flows generated by a convex quadratic loss and a general convex lower semicontinuous mirror potential. We show that, when initialized near the boundary of the domain of the mirror potential, their rescaled trajectories converge to a limiting mirror flow whose potential is the indicator function of the domain. In this limit, the primal variable minimizes the loss over a time-dependent hypothesis set: the subdifferential of the support function of the domain, evaluated at the dual variable. This characterization provides a general mechanism for incremental learning in mirror flows.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 21 of its 34 references that are in the map. First posted on arXiv 22 Jun 2026.
Builds on
21 of 34 references.
References · 21
- 2017A Closer Look at Memorization in Deep Networkscited by 942
- 2013Exact solutions to the nonlinear dynamics of learning in deep linear neural networkscited by 763
- 2019Implicit Regularization in Deep Matrix Factorizationcited by 405
- 2017Implicit Regularization in Matrix Factorizationcited by 322
- 2020Kernel and Rich Regimes in Overparametrized Modelscited by 306
- 2019SGD on Neural Networks Learns Functions of Increasing Complexitycited by 204
- 2020Towards Resolving the Implicit Bias of Gradient Descent for Matrix Factorization: Greedy Low-Rank Learningcited by 150
- 2023SGD learning on neural networks: leap complexity and saddle-to-saddle dynamicscited by 143
- 2019Implicit Regularization of Discrete Gradient Dynamics in Linear Neural Networkscited by 124
- 2020Implicit Regularization in Deep Learning May Not Be Explainable by Normscited by 119
- 2021Saddle-to-Saddle Dynamics in Deep Linear Networks: Small Initialization Training, Symmetry, and Sparsitycited by 85
- 2022Gradient flow dynamics of shallow ReLU networks for square loss and orthogonal inputscited by 76
9 more
- 2023Saddle-to-Saddle Dynamics in Diagonal Linear Networkscited by 57
- 2023On Learning Gaussian Multi-index Models with Gradient Flowcited by 57
- 2022Implicit Bias of Gradient Descent on Reparametrized Models: On Equivalence to Mirror Descentcited by 39
- 2022Incremental Learning in Diagonal Linear Networkscited by 28
- 2025Learning In-context n-grams with Transformers: Sub-n-grams Are Near-stationary Pointscited by 16
- 2025Saddle-to-Saddle Dynamics Explains A Simplicity Bias Across Neural Network Architecturescited by 15
- 2014Sparse Recovery via Differential Inclusionscited by 11
- 2023Implicit Regularization for Group Sparsitycited by 8
- 2025Diagonal Linear Networks and the Lasso Regularization Pathcited by 4
Built on by
Citing papers per year: 2026: 2.
2 papers.