mlsanityData to 14 Oct 2026

How Much Orthogonalization Does Muon Need?

arXiv 2606.00371 ↗First posted 29 May 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Muon optimizers improve neural-network training by replacing ill-conditioned momentum updates with approximately semi-orthogonal updates. This motivates a practical question: how much orthogonalization does Muon actually require? We study this question using a relaxed cubic Newton--Schulz schedule derived directly for Muon's low precision singular value band. The resulting five-step cubic construction uses ten dominant matrix multiplications, compared with fifteen for five quintic Newton--Schulz iterations. The cubic schedule is not intended as a more accurate polar solver; instead, it is a principled low-cost variant that lets us probe the relation between polar accuracy, spectral shaping, and training quality. Across synthetic diagnostics, NanoGPT ablations, and training experiments on hybrid MoE/Mamba models, we find that training quality is not governed monotonically by polar-decomposition accuracy: truncated Polar Express, Muon-Jordan, cubic Newton--Schulz, and an explicit FP32 SVD polar factor can reach nearly indistinguishable final loss on GPT-2 Small, and cubic5 matches the Muon-Jordan quintic update within about 10-3 validation loss on hybrid MoE/Mamba models with one billion to four billion parameters. These results support cubic5 as a practical low-cost Muon orthogonalization variant, with empirical evidence of training-quality parity in the settings tested.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 1 of its 12 references that are in the map. First posted on arXiv 29 May 2026.

Builds on

1 of 12 references.

References · 1

  1. 2025The Polar Express: Optimal Matrix Sign Methods and Their Application to the Muon Algorithmcited by 110

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Training-Loss Guarantees for Muon with Finite-Step Newton--Schulz Orthogonalization
  2. 2026Muon2: Boosting Muon via Adaptive Second-Moment Preconditioning
  3. 2025Muon is Scalable for LLM Training
  4. 2026Orthogonal Witness Control for Muon Optimization via Sigmoid Spectral Reshaping
  5. 2026An Isotropy-Preserving Spectral Cap for Muon: Theory and Three Case Studies
  6. 2025Effective Quantization of Muon Optimizer States
  7. 2026Muon-C: Operator-Aligned Muon for Convolutional Kernels
  8. 2026Move on Muon : A Hamiltonian probability gradient flow perspective of Muon optimizer