# Beyond the Ideal: Analyzing the Inexact Muon Update

arXiv: https://arxiv.org/abs/2510.19933 · HTML: https://mlsanity.com/paper/2510.19933/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 42 references that are in the map.
- First posted on arXiv 22 Oct 2025.

## Builds on (most cited first, 26 of 26)

- [The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale](https://mlsanity.com/paper/2406.17557/) (2024)
- [Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programming](https://mlsanity.com/paper/1309.5549/) (2013)
- [Muon is Scalable for LLM Training](https://mlsanity.com/paper/2502.16982/) (2025)
- [Why gradient clipping accelerates training: A theoretical justification for adaptivity](https://mlsanity.com/paper/1905.11881/) (2019)
- [Lower Bounds for Non-Convex Stochastic Optimization](https://mlsanity.com/paper/1912.02365/) (2019)
- [Old Optimizer, New Norm: An Anthology](https://mlsanity.com/paper/2409.20325/) (2024)
- [Training Deep Learning Models with Norm-Constrained LMOs](https://mlsanity.com/paper/2502.07529/) (2025)
- [Momentum Improves Normalized SGD](https://mlsanity.com/paper/2002.03305/) (2020)
- [Lower Bounds for Finding Stationary Points I](https://mlsanity.com/paper/1710.11606/) (2017)
- [The Polar Express: Optimal Matrix Sign Methods and Their Application to the Muon Algorithm](https://mlsanity.com/paper/2505.16932/) (2025)
- [Fantastic Pretraining Optimizers and Where to Find Them](https://mlsanity.com/paper/2509.02046/) (2025)
- [Understanding Gradient Orthogonalization for Deep Learning via Non-Euclidean Trust-Region Optimization](https://mlsanity.com/paper/2503.12645/) (2025)
- [Muon Optimizes Under Spectral Norm Constraints](https://mlsanity.com/paper/2506.15054/) (2025)
- [A Note on the Convergence of Muon](https://mlsanity.com/paper/2502.02900/) (2025)
- [Practical Efficiency of Muon for Pretraining](https://mlsanity.com/paper/2505.02222/) (2025)
- [Gluon: Making Muon & Scion Great Again! (Bridging Theory and Practice of LMO-based Optimizers for LLMs)](https://mlsanity.com/paper/2505.13416/) (2025)
- [MuonBP: Faster Muon via Block-Periodic Orthogonalization](https://mlsanity.com/paper/2510.16981/) (2025)
- [Orthogonalising gradients to speed up neural network optimisation](https://mlsanity.com/paper/2202.07052/) (2022)
- [Two Sides of One Coin: the Limits of Untuned SGD and the Power of Adaptive Methods](https://mlsanity.com/paper/2305.12475/) (2023)
- [Optimizing (L0, L1)-Smooth Functions by Gradient Methods](https://mlsanity.com/paper/2410.10800/) (2024)
- [Generalized Gradient Norm Clipping & Non-Euclidean (L0,L1)-Smoothness](https://mlsanity.com/paper/2506.01913/) (2025)
- [Error Feedback for Muon and Friends](https://mlsanity.com/paper/2510.00643/) (2025)
- [Drop-Muon: Update Less, Converge Faster](https://mlsanity.com/paper/2510.02239/) (2025)
- [Mirror Descent Under Generalized Smoothness](https://mlsanity.com/paper/2502.00753/) (2025)
- [The Ball-Proximal (="Broximal") Point Method: a New Algorithm, Convergence Theory, and Applications](https://mlsanity.com/paper/2502.02002/) (2025)
- [Non-Euclidean Broximal Point Method: A Blueprint for Geometry-Aware Optimization](https://mlsanity.com/paper/2510.00823/) (2025)

## Built on by (most cited first, 27 of 27)

- [MuonEq: Balancing Before Orthogonalization with Lightweight Equilibration](https://mlsanity.com/paper/2603.28254/) (2026)
- [Insights on Muon from Simple Quadratics](https://mlsanity.com/paper/2602.11948/) (2026)
- [Deriving Hyperparameter Scaling Laws via Modern Optimization Theory](https://mlsanity.com/paper/2603.15958/) (2026)
- [Non-Euclidean SGD for Structured Optimization: Unified Analysis and Improved Rates](https://mlsanity.com/paper/2511.11466/) (2025)
- [Muon with Nesterov Momentum: Heavy-Tailed Noise and (Randomized) Inexact Polar Decomposition](https://mlsanity.com/paper/2605.06884/) (2026)
- [Why Muon Outperforms Adam: A Curvature Perspective](https://mlsanity.com/paper/2606.04662/) (2026)
- [Muon with Finite Newton-Schulz: The Smoothing Benefit in Nonsmooth Nonconvex Optimization](https://mlsanity.com/paper/2608.26288/) (2026)
- [Ky Fan Norms and Beyond: Dual Norms and Combinations for Matrix Optimization](https://mlsanity.com/paper/2512.09678/) (2025)
- [Beyond Muon: MUD (MomentUm Decorrelation) for Faster Transformer Training](https://mlsanity.com/paper/2603.17970/) (2026)
- [Communication-Efficient Gluon in Federated Learning](https://mlsanity.com/paper/2604.10689/) (2026)
- [Phases of Muon: When Muon Eclipses SignSGD](https://mlsanity.com/paper/2605.09552/) (2026)
- [Muon Learns More Robust and Transferable Features than Adam](https://mlsanity.com/paper/2606.09658/) (2026)
- [CacheMuon: Using Temporal Preconditioning To Approximate Polar Factor](https://mlsanity.com/paper/2606.16371/) (2026)
- [Ringmaster LMO: Asynchronous Linear Minimization Oracle Momentum Method](https://mlsanity.com/paper/2605.18174/) (2026)
- [Turbo-Muon: Almost-Orthogonal Pre-Conditioning for Fast Muon Updates](https://mlsanity.com/paper/2512.04632/) (2025)
- [When and Why Grouping Attention Heads Accelerates Muon Optimization](https://mlsanity.com/paper/2605.08933/) (2026)
- [A Note on Stability for Orthogonalized Matrix Momentum with Client Sampling](https://mlsanity.com/paper/2606.01720/) (2026)
- [Schattor: Schatten-family methods for deep learning optimization](https://mlsanity.com/paper/2606.15702/) (2026)
- [Reassessing Muon for Matrix Factorization](https://mlsanity.com/paper/2607.13246/) (2026)
- [Muse: Representation Geometry of Muon Beyond Normalized Momentum](https://mlsanity.com/paper/2607.14536/) (2026)
- [Constrained Stochastic Spectral Preconditioning Converges for Nonconvex Objectives](https://mlsanity.com/paper/2605.11850/) (2026)
- [Sign compression for Muon: SignMuon, MuonSign, and the Limits of Error Feedback](https://mlsanity.com/paper/2607.29674/) (2026)
- [Nonsmooth Optimization via Orthogonalized Momentum](https://mlsanity.com/paper/2609.13677/) (2026)
- [Cost-free Spectral Estimation for Adaptive Newton--Schulz in Matrix Optimizers](https://mlsanity.com/paper/2609.33047/) (2026)
- [Muon Learns Facts Better: Understanding the Role of Spectral Orthogonalization](https://mlsanity.com/paper/2610.02798/) (2026)
- [ExpertMuon-Compass: Alignment-Guided Step Sizes for Mixture-of-Experts Training](https://mlsanity.com/paper/2610.04140/) (2026)
- [Randomized Momentum Transport with Cheap Predictors](https://mlsanity.com/paper/2610.13789/) (2026)

## Abstract (arXiv)

The Muon optimizer has rapidly emerged as a powerful, geometry-aware alternative to AdamW, demonstrating strong performance in large-scale training of neural networks. However, a critical theory-practice disconnect exists: Muon's efficiency relies on fast, approximate orthogonalization, yet all prior theoretical work analyzes an idealized, computationally intractable version assuming exact SVD-based updates. This work moves beyond the ideal by providing the first analysis of the inexact orthogonalized update at Muon's core. We develop our analysis within the general framework of Linear Minimization Oracle (LMO)-based optimization, introducing a realistic additive error model to capture the inexactness of practical approximation schemes. Our analysis yields explicit bounds that quantify performance degradation as a function of the LMO inexactness/error. We reveal a fundamental coupling between this inexactness and the optimal step size and momentum: lower oracle precision requires a smaller step size but larger momentum parameter. These findings elevate the approximation procedure (e.g., the number of Newton-Schulz steps) from an implementation detail to a critical parameter that must be co-tuned with the learning schedule. NanoGPT experiments directly confirm the predicted coupling, with optimal learning rates clearly shifting as approximation precision changes.