# The Spectral Dynamics and Noise Geometry of Muon

arXiv: https://arxiv.org/abs/2606.08388 · HTML: https://mlsanity.com/paper/2606.08388/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 41 references that are in the map.
- First posted on arXiv 7 Jun 2026.

## Builds on (most cited first, 30 of 30)

- [The Implicit Bias of Gradient Descent on Separable Data](https://mlsanity.com/paper/1710.10345/) (2017)
- [Implicit Regularization in Matrix Factorization](https://mlsanity.com/paper/1705.09280/) (2017)
- [Don't Decay the Learning Rate, Increase the Batch Size](https://mlsanity.com/paper/1711.00489/) (2017)
- [An Empirical Model of Large-Batch Training](https://mlsanity.com/paper/1812.06162/) (2018)
- [Characterizing Implicit Bias in Terms of Optimization Geometry](https://mlsanity.com/paper/1802.08246/) (2018)
- [Understanding Gradient Orthogonalization for Deep Learning via Non-Euclidean Trust-Region Optimization](https://mlsanity.com/paper/2503.12645/) (2025)
- [A Note on the Convergence of Muon](https://mlsanity.com/paper/2502.02900/) (2025)
- [Practical Efficiency of Muon for Pretraining](https://mlsanity.com/paper/2505.02222/) (2025)
- [PolarGrad: A Class of Matrix-Gradient Optimizers from a Unifying Preconditioning Perspective](https://mlsanity.com/paper/2505.21799/) (2025)
- [Implicit Bias of Spectral Descent and Muon on Multiclass Separable Data](https://mlsanity.com/paper/2502.04664/) (2025)
- [Approaching Deep Learning through the Spectral Dynamics of Weights](https://mlsanity.com/paper/2408.11804/) (2024)
- [Convergence Bound and Critical Batch Size of Muon Optimizer](https://mlsanity.com/paper/2507.01598/) (2025)
- [Orthogonalising gradients to speed up neural network optimisation](https://mlsanity.com/paper/2202.07052/) (2022)
- [Critical Batch Size Revisited: A Simple Empirical Approach to Large-Batch Language Model Training](https://mlsanity.com/paper/2505.23971/) (2025)
- [How Muon's Spectral Design Benefits Generalization: A Study on Imbalanced Data](https://mlsanity.com/paper/2510.22980/) (2025)
- [Muon2: Boosting Muon via Adaptive Second-Moment Preconditioning](https://mlsanity.com/paper/2604.09967/) (2026)
- [Muon is Not That Special: Random or Inverted Spectra Work Just as Well](https://mlsanity.com/paper/2605.11181/) (2026)
- [Implicit Regularization in Matrix Sensing via Mirror Descent](https://mlsanity.com/paper/2105.13831/) (2021)
- [The Implicit Bias of Adam and Muon on Smooth Homogeneous Neural Networks](https://mlsanity.com/paper/2602.16340/) (2026)
- [Uniform Spectral Growth and Convergence of Muon in LoRA-Style Matrix Factorization](https://mlsanity.com/paper/2602.06385/) (2026)
- [Muon Does Not Converge on Convex Lipschitz Functions](https://mlsanity.com/paper/2605.08980/) (2026)
- [Rethinking Muon Beyond Pretraining: Spectral Failures and High-Pass Remedies for VLA and RLVR](https://mlsanity.com/paper/2605.19282/) (2026)
- [TrasMuon: Trust-Region Adaptive Scaling for Orthogonalized Momentum Optimizers](https://mlsanity.com/paper/2602.13498/) (2026)
- [Spectral Dynamics in Deep Networks: Feature Learning, Outlier Escape, and Learning Rate Transfer](https://mlsanity.com/paper/2605.07870/) (2026)
- [Spectral Flattening Is All Muon Needs: How Orthogonalization Controls Learning Rate and Convergence](https://mlsanity.com/paper/2605.13079/) (2026)
- [AMUSE: Anytime Muon with Stable Gradient Evaluation](https://mlsanity.com/paper/2605.22432/) (2026)
- [pAI/MSc: ML Theory Research with Humans on the Loop](https://mlsanity.com/paper/2604.20622/) (2026)
- [DP-Muon: Differentially Private Optimization via Matrix-Orthogonalized Momentum](https://mlsanity.com/paper/2605.12994/) (2026)
- [OrScale: Orthogonalised Optimization with Layer-Wise Trust-Ratio Scaling](https://mlsanity.com/paper/2605.07815/) (2026)
- [When Muon Optimizer Meets Adversarial Training: A Theoretical and Empirical Study](https://mlsanity.com/paper/2605.26929/) (2026)

## Built on by (most cited first, 3 of 3)

- [The Loss Does Not See the Basis, but Adam Does](https://mlsanity.com/paper/2608.05136/) (2026)
- [Physical Muon: Orthogonalization as an Equilibrium Computation](https://mlsanity.com/paper/2609.37525/) (2026)
- [Does Muon Need Fine-Grained Spectral Shaping?](https://mlsanity.com/paper/2610.07497/) (2026)

## Abstract (arXiv)

Muon replaces a matrix gradient G=USigma V^top by its polar factor UV^top. This keeps the singular directions selected by the gradient, but makes the update spectrum flat. We study the optimization bias created by this operation. Under explicit alignment assumptions, we prove that the polar update is the one-step entropy-maximizing choice among bounded updates that use the gradient singular directions and do not adapt to the current weight spectrum. In an underdetermined regression model, we derive exact singular-value dynamics for continuous-time Muon and identify a measurement-dependent condition under which the normalized spectrum moves toward equal nonzero singular values. This geometry also rules out a common low-rank interpretation: at fixed Frobenius norm, Muon's distinguished state has a flat spectrum, whereas nuclear-norm minimization favors spectral concentration. Controlled matrix-sensing experiments separate the effect from simple gradient rescaling, show that norm-matched gradient descent does not reproduce Muon, and recover the predicted flattening trend across broad ablations. In small NanoGPT pretraining, Muon preserves stable rank, has a broad learning-rate plateau, and improves validation loss relative to AdamW; in a matched small-ViT control, the ranking reverses. The resulting picture is regime-dependent: Muon is not universally superior, but its flat-spectrum bias can help when many spectral directions need to remain active.