mlsanityData to 14 Oct 2026

Improved Convergence Rates of Muon Optimizer for Nonconvex Optimization

arXiv 2601.19400 ↗First posted 27 Jan 2026math.OC cs.LGcited by 9

Not on the map. Why?

Abstract (arXiv)

The Muon optimizer has recently attracted attention due to its orthogonalized first-order updates, and a deeper theoretical understanding of its convergence behavior is essential for guiding practical applications; however, existing convergence guarantees are either coarse or obtained under restrictive analytical settings. In this work, we establish sharper convergence guarantees for the Muon optimizer through a direct and simplified analysis that does not rely on restrictive assumptions on the update rule. Our results improve upon existing bounds by achieving faster convergence rates while covering a broader class of problem settings. These findings provide a more accurate theoretical characterization of Muon and offer insights applicable to a broader class of orthogonalized first-order methods.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 14 of its 24 references that are in the map. First posted on arXiv 27 Jan 2026.

Builds on

14 of 24 references.

References · 14

  1. 2017Don't Decay the Learning Rate, Increase the Batch Sizecited by 320
  2. 2025Training Deep Learning Models with Norm-Constrained LMOscited by 192
  3. 2025A Note on the Convergence of Muoncited by 83
  4. 2025Muon Outperforms Adam in Tail-End Associative Memory Learningcited by 46
  5. 2025On the Convergence of Muon and Beyondcited by 31
  6. 2025Convergence Bound and Critical Batch Size of Muon Optimizercited by 28
  7. 2025AdaGrad Meets Muon: Adaptive Stepsizes for Orthogonal Updatescited by 23
  8. 2025LiMuon: Light and Fast Muon Optimizer for Large Modelscited by 22
  9. 2025Generalized Gradient Norm Clipping & Non-Euclidean (L0,L1)-Smoothnesscited by 17
  10. 2025FedMuon: Accelerating Federated Learning with Matrix Orthogonalizationcited by 16
  11. 2025Dion2: A Simple Method to Shrink Matrix in Muoncited by 13
  12. 2024Increasing Both Batch Size and Learning Rate Accelerates Stochastic Gradient Descentcited by 11
2 more
  1. 2025A Convergence Analysis of Adaptive Optimizers under Floating-point Quantizationcited by 8
  2. 2025Faster Convergence of Riemannian Stochastic Gradient Descent with Increasing Batch Sizecited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 9.

9 papers.

  1. 2026MuonEq: Balancing Before Orthogonalization with Lightweight Equilibrationcited by 17
  2. 2026Insights on Muon from Simple Quadraticscited by 13
  3. 2026Muon Converges under Heavy-Tailed Noise: Nonconvex H"older-Smooth Empirical Risk Minimizationcited by 2
  4. 2026LionMuon: Alternating Spectral and Sign Descent for Efficient Trainingcited by 2
  5. 2026Reassessing Muon for Matrix Factorizationcited by 1
  6. 2026Muse: Representation Geometry of Muon Beyond Normalized Momentumcited by 1
  7. 2026MALT: Lightweight Curvature-Aware Muon via Diagonal Preconditioningcited by 1
  8. 2026Accelerating LMO-Based Optimization via Implicit Gradient Transportcited by 0
  9. 2026LionMuon: Alternating Spectral and Sign Descent for Efficient Trainingcited by 0