mlsanityData to 7 Oct 2026

PolarAdamW: Disentangling Spectral Control and Schur Gauge-Equivariance in Matrix Optimisation

arXiv 2605.07067 ↗First posted 8 May 2026cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

Muon's matrix-level update couples two distinct effects: spectral control via a polar map, and equivariance under orthogonal changes of multiplicity-space basis (Schur gauge-equivariance). We separate them with PolarAdamW, a controlled hybrid that preserves Muon's polar spectral-norm control but breaks the gauge-equivariance, since AdamW's coordinatewise preconditioner is basis-dependent. Algorithmically, PolarAdamW applies Muon's Newton-Schulz polar map to AdamW's preconditioned direction rather than to raw momentum, at per-iteration wall-time comparable to Muon. We prove that Muon's polar step is Schur gauge-equivariant on multiplicity matrices while AdamW's coordinatewise step is not. On DeiT-Tiny trained from scratch on four independently sampled 100-class subsets of ImageNet-1k, where multiplicity-basis freedom is trivial, PolarAdamW outperforms Muon by +1.93 pp in test accuracy on average and AdamW by +9.5 pp; under the 300-epoch DeiT-style recipe, it remains ahead of Muon by +1.37 pp and AdamW by +5.80 pp on average. On SO(3)-equivariant 3D point-cloud regression, where multiplicity-basis freedom is non-trivial, the ordering reverses: Muon outperforms PolarAdamW at every audited capacity, and the gap widens with capacity. Both matrix-polar optimisers continue to outperform AdamW. This double dissociation separates spectral control from Schur gauge-equivariance: the first composes well with AdamW preconditioning on standard transformers, while the second becomes consequential when multiplicity-basis freedom is structurally non-trivial.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 19 of its 25 references that are in the map. First posted on arXiv 8 May 2026.

Builds on

19 of 25 references.

References · 19

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  3. 2016Group Equivariant Convolutional Networkscited by 1,154
  4. 2021E(3)-Equivariant Graph Neural Networks for Data-Efficient and Accurate Interatomic Potentialscited by 701
  5. 2022MACE: Higher Order Equivariant Message Passing Neural Networks for Fast and Accurate Force Fieldscited by 467
  6. 2019General E(2)-Equivariant Steerable CNNscited by 441
  7. 2018Shampoo: Preconditioned Stochastic Tensor Optimizationcited by 413
  8. 2025The Polar Express: Optimal Matrix Sign Methods and Their Application to the Muon Algorithmcited by 104
  9. 2025NorMuon: Making Muon more efficient and scalablecited by 74
  10. 2025AdaMuon: Adaptive Muon Optimizercited by 59
  11. 2025PolarGrad: A Class of Matrix-Gradient Optimizers from a Unifying Preconditioning Perspectivecited by 48
  12. 2023Optimization Dynamics of Equivariant and Augmented Neural Networkscited by 17
7 more
  1. 2026MuonEq: Balancing Before Orthogonalization with Lightweight Equilibrationcited by 16
  2. 2026Mousse: Rectifying the Geometry of Muon with Curvature-Aware Preconditioningcited by 15
  3. 2026Muon2: Boosting Muon via Adaptive Second-Moment Preconditioningcited by 15
  4. 2024Understanding Adam Requires Better Rotation Dependent Assumptionscited by 10
  5. 2026FISMO: Fisher-Structured Momentum-Orthogonalized Optimizercited by 10
  6. 2025Adam or Gauss-Newton? A Comparative Study In Terms of Basis Alignment and SGD Noisecited by 2
  7. 2026NS-RGS: Newton-Schulz based Riemannian gradient method for orthogonal group synchronizationcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.