# Muon Dynamics as a Spectral Wasserstein Flow

arXiv: https://arxiv.org/abs/2604.04891 · HTML: https://mlsanity.com/paper/2604.04891/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 29 references that are in the map.
- First posted on arXiv 6 Apr 2026.

## Builds on (most cited first, 16 of 16)

- [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018)
- [A Mean Field View of the Landscape of Two-Layers Neural Networks](https://mlsanity.com/paper/1804.06561/) (2018)
- [Shampoo: Preconditioned Stochastic Tensor Optimization](https://mlsanity.com/paper/1802.09568/) (2018)
- [On the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal Transport](https://mlsanity.com/paper/1805.09545/) (2018)
- [Generative Moment Matching Networks](https://mlsanity.com/paper/1502.02761/) (2015)
- [Training Deep Learning Models with Norm-Constrained LMOs](https://mlsanity.com/paper/2502.07529/) (2025)
- [Optimal Entropy-Transport problems and a new Hellinger-Kantorovich distance between positive measures](https://mlsanity.com/paper/1508.07941/) (2015)
- [Momentum Improves Normalized SGD](https://mlsanity.com/paper/2002.03305/) (2020)
- [Subspace Robust Wasserstein Distances](https://mlsanity.com/paper/1901.08949/) (2019)
- [Revisiting Normalized Gradient Descent: Fast Evasion of Saddle Points](https://mlsanity.com/paper/1711.05224/) (2017)
- [Wasserstein Discriminant Analysis](https://mlsanity.com/paper/1608.08063/) (2016)
- [Existence, Duality, and Cyclical monotonicity for weak transport costs](https://mlsanity.com/paper/1809.05893/) (2018)
- [Structured Transforms Across Spaces with Cost-Regularized Optimal Transport](https://mlsanity.com/paper/2311.05788/) (2023)
- [Matrix-valued Monge-Kantorovich Optimal Mass Transport](https://mlsanity.com/paper/1304.3931/) (2013)
- [Matrix Optimal Mass Transport: A Quantum Mechanical Approach](https://mlsanity.com/paper/1610.03041/) (2016)
- [Covariance-modulated optimal transport and gradient flows](https://mlsanity.com/paper/2302.07773/) (2023)

## Built on by (most cited first, 6 of 6)

- [Phases of Muon: When Muon Eclipses SignSGD](https://mlsanity.com/paper/2605.09552/) (2026)
- [Move on Muon : A Hamiltonian probability gradient flow perspective of Muon optimizer](https://mlsanity.com/paper/2605.23871/) (2026)
- [From SGD to Muon: Adaptive Optimization via Schatten-p Norms](https://mlsanity.com/paper/2605.19781/) (2026)
- [Muon as a Residual Connection](https://mlsanity.com/paper/2607.01124/) (2026)
- [Physical Muon: Orthogonalization as an Equilibrium Computation](https://mlsanity.com/paper/2609.37525/) (2026)
- [ORCA: The Annealed Spectral Conditioning Optimizer for Faster, Better LLM Training](https://mlsanity.com/paper/2610.06116/) (2026)

## Abstract (arXiv)

Gradient normalization stabilizes deep-learning optimization, and spectral normalizations are especially natural for matrix-shaped parameter blocks; Muon is the motivating example. We study an idealized deterministic, continuous-time, vanishing-momentum version of this idea in the mean-field regime, where wide models are represented by probability measures on parameter space. Starting from normalized matrix flows, we introduce Spectral Wasserstein distances indexed by norms γ on positive semidefinite matrices: the trace norm gives classical W2, the operator norm gives the Muon geometry, and Schatten norms interpolate between them. We develop the static Kantorovich formulation, a max-min robust-cost representation, Gaussian reductions extending the Bures formula, and for monotone norms, prove equivalence with a Benamou--Brenier formulation. This yields a gradient-flow interpretation of the mean-field normalized training dynamics. We illustrate these findings by numerical experiments on MMD flows, Gaussian reductions, two-layer ReLU models, and shallow attention.