mlsanityData to 14 Oct 2026

MuonAll: Muon Variant for Efficient Finetuning of Large Language Models

arXiv 2511.06086 ↗First posted 8 Nov 2025cs.CL cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

Muon optimizer has demonstrated robust results in pretraining of language models but its performance in finetuning of existing public pretrained models is not yet explored. Currently, Muon is used along with AdamW introducing a scope of improvement for adopting all parameters inside Muon. We introduce MuonAll, which incorporates all the parameters inside Muon by transforming into 2D matrices. We conduct extensive finetuning experiments across publicly available language models with model sizes upto half billion parameters. Muon and MuonAll perform at par with AdamW across major benchmarks, highlighting their effectiveness as alternative optimizers. We open-source the distributed implementations of Muon and MuonAll, available at https://github.com/Saurabh750/optimizer

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 25 of its 36 references that are in the map. First posted on arXiv 8 Nov 2025.

Builds on

25 of 36 references.

References · 25

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  4. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  5. 2024GPT-4o System Cardcited by 4,588
  6. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  7. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  8. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
  9. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,183
  10. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,164
  11. 2024Qwen2 Technical Reportcited by 2,120
  12. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,135
13 more
  1. 2025Muon is Scalable for LLM Trainingcited by 428
  2. 2025SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Modelcited by 335
  3. 2023Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-trainingcited by 265
  4. 2024Old Optimizer, New Norm: An Anthologycited by 193
  5. 2018On the Convergence of Stochastic Gradient Descent with Adaptive Stepsizescited by 156
  6. 2025Muon Optimizes Under Spectral Norm Constraintscited by 86
  7. 2025A Note on the Convergence of Muoncited by 83
  8. 2025Practical Efficiency of Muon for Pretrainingcited by 80
  9. 2025AdaMuon: Adaptive Muon Optimizercited by 67
  10. 2025Convergence Bound and Critical Batch Size of Muon Optimizercited by 28
  11. 2025MuLoCo: Muon is a practical inner optimizer for DiLoCocited by 18
  12. 2025A Stable Whitening Optimizer for Efficient Neural Network Trainingcited by 18
  13. 2024Curvature-Informed SGD via General Purpose Lie-Group Preconditionerscited by 16

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 7.

7 papers.

  1. 2026MuonEq: Balancing Before Orthogonalization with Lightweight Equilibrationcited by 17
  2. 2026Spectral Flattening Is All Muon Needs: How Orthogonalization Controls Learning Rate and Convergencecited by 5
  3. 2026OLion: Approaching the Hadamard Ideal by Intersecting Spectral and ℓ∞ Implicit Biasescited by 3
  4. 2026When and Why SignSGD Outperforms SGD: A Theoretical Study Based on ℓ1-norm Lower Boundscited by 2
  5. 2026Tight Complexity of Muon under Heavy-Tailed Noisecited by 2
  6. 2026IFNSO: Iteration-Free Newton-Schulz Orthogonalizationcited by 1
  7. 2026AF-Muon: An AdamW-Free Muon Optimizer for Tied-Embedding Modelscited by 0