PolarAdamW: Disentangling Spectral Control and Schur Gauge-Equivariance in Matrix Optimisation
Not on the map. Why?
Abstract (arXiv)
Muon's matrix-level update couples two distinct effects: spectral control via a polar map, and equivariance under orthogonal changes of multiplicity-space basis (Schur gauge-equivariance). We separate them with PolarAdamW, a controlled hybrid that preserves Muon's polar spectral-norm control but breaks the gauge-equivariance, since AdamW's coordinatewise preconditioner is basis-dependent. Algorithmically, PolarAdamW applies Muon's Newton-Schulz polar map to AdamW's preconditioned direction rather than to raw momentum, at per-iteration wall-time comparable to Muon. We prove that Muon's polar step is Schur gauge-equivariant on multiplicity matrices while AdamW's coordinatewise step is not. On DeiT-Tiny trained from scratch on four independently sampled 100-class subsets of ImageNet-1k, where multiplicity-basis freedom is trivial, PolarAdamW outperforms Muon by +1.93 pp in test accuracy on average and AdamW by +9.5 pp; under the 300-epoch DeiT-style recipe, it remains ahead of Muon by +1.37 pp and AdamW by +5.80 pp on average. On SO(3)-equivariant 3D point-cloud regression, where multiplicity-basis freedom is non-trivial, the ordering reverses: Muon outperforms PolarAdamW at every audited capacity, and the gap widens with capacity. Both matrix-polar optimisers continue to outperform AdamW. This double dissociation separates spectral control from Schur gauge-equivariance: the first composes well with AdamW preconditioning on standard transformers, while the second becomes consequential when multiplicity-basis freedom is structurally non-trivial.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 19 of its 25 references that are in the map. First posted on arXiv 8 May 2026.
Builds on
19 of 25 references.
References · 19
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
- 2016Group Equivariant Convolutional Networkscited by 1,154
- 2021E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentialscited by 701
- 2022MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fieldscited by 467
- 2019General E(2)-Equivariant Steerable CNNscited by 441
- 2018Shampoo: Preconditioned Stochastic Tensor Optimizationcited by 413
- 2025The Polar Express: Optimal Matrix Sign Methods and Their Application to the Muon Algorithmcited by 104
- 2025NorMuon: Making Muon more efficient and scalablecited by 74
- 2025AdaMuon: Adaptive Muon Optimizercited by 59
- 2025PolarGrad: A Class of Matrix-Gradient Optimizers from a Unifying Preconditioning Perspectivecited by 48
- 2023Optimization Dynamics of Equivariant and Augmented Neural Networkscited by 17
7 more
- 2026MuonEq: Balancing Before Orthogonalization with Lightweight Equilibrationcited by 16
- 2026Mousse: Rectifying the Geometry of Muon with Curvature-Aware Preconditioningcited by 15
- 2026Muon2: Boosting Muon via Adaptive Second-Moment Preconditioningcited by 15
- 2024Understanding Adam Requires Better Rotation Dependent Assumptionscited by 10
- 2026FISMO: Fisher-Structured Momentum-Orthogonalized Optimizercited by 10
- 2025Adam or Gauss-Newton? A Comparative Study In Terms of Basis Alignment and SGD Noisecited by 2
- 2026NS-RGS: Newton-Schulz based Riemannian gradient method for orthogonal group synchronizationcited by 2
Built on by
Citing papers per year: 2026: 2.
2 papers.
- 2026The Loss Does Not See the Basis, but Adam Doescited by 0
- 2026Equivariance Breaks the Learning Ratecited by 0
- 2026Equivariance Breaks the Learning Rate8 Sep 2026
- 2026The Loss Does Not See the Basis, but Adam Does5 Aug 2026