mlsanityData to 7 Oct 2026

Mirrorless Mirror Descent: A Natural Derivation of Mirror Descent

arXiv 2004.01025 ↗First posted 2 Apr 2020cs.LG math.OC stat.MLcited by 18

Not on the map. Why?

Abstract (arXiv)

We present a primal only derivation of Mirror Descent as a "partial" discretization of gradient flow on a Riemannian manifold where the metric tensor is the Hessian of the Mirror Descent potential. We contrast this discretization to Natural Gradient Descent, which is obtained by a "full" forward Euler discretization. This view helps shed light on the relationship between the methods and allows generalizing Mirror Descent to general Riemannian geometries, even when the metric tensor is em not a Hessian, and thus there is no "dual."

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). First posted on arXiv 2 Apr 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 6; 2023: 3; 2024: 2; 2025: 3; 2026: 1.

18 papers.

Sort:

  1. 2022Understanding Gradient Descent on Edge of Stability in Deep Learningcited by 126
  2. 2024Get rich quick: exact solutions reveal how unbalanced initializations promote rapid feature learningcited by 41
  3. 2021Finite Sample Analysis of Two-Time-Scale Natural Actor-Critic Algorithmcited by 39
  4. 2022Implicit Bias of Gradient Descent on Reparametrized Models: On Equivalence to Mirror Descentcited by 38
  5. 2021More is Less: Inducing Sparsity via Overparameterizationcited by 22
  6. 2022Smooth over-parameterized solvers for non-smooth structured optimizationcited by 11
  7. 2022Non-convex online learning via algorithmic equivalencecited by 10
  8. 2023Stochastic Approximation Beyond Gradient for Signal Processing and Machine Learningcited by 10
  9. 2022From Gradient Flow on Population Loss to Learning with Stochastic Gradient Descentcited by 5
  10. 2021Approximate Newton policy gradient algorithmscited by 3
  11. 2025Mirror Descent and Novel Exponentiated Gradient Algorithms Using Trace-Form Entropies and Deformed Logarithmscited by 3
  12. 2024A Mirror Descent Perspective of Smoothed Sign Descentcited by 2
6 more
  1. 2025Mirror Descent Using the Tempesta Generalized Multi-parametric Logarithmscited by 1
  2. 2022Iterative regularization in classification via hinge loss diagonal descentcited by 0
  3. 2023Local Convergence of Gradient Methods for Min-Max Games: Partial Curvature Generically Sufficescited by 0
  4. 2023A Challenge in Reweighting Data with Bilevel Optimizationcited by 0
  5. 2025Thermodynamic structure of the Sinkhorn flowcited by 0
  6. 2026Mirror Descent on Riemannian Manifoldscited by 0