# SignMuon: Communication-Efficient Distributed Muon Optimization

arXiv: https://arxiv.org/abs/2605.16311 · HTML: https://mlsanity.com/paper/2605.16311/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 35 references that are in the map.
- First posted on arXiv 4 May 2026.

## Builds on (most cited first, 14 of 14)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Optimizing Neural Networks with Kronecker-factored Approximate Curvature](https://mlsanity.com/paper/1503.05671/) (2015)
- [Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training](https://mlsanity.com/paper/1712.01887/) (2017)
- [Symbolic Discovery of Optimization Algorithms](https://mlsanity.com/paper/2302.06675/) (2023)
- [Shampoo: Preconditioned Stochastic Tensor Optimization](https://mlsanity.com/paper/1802.09568/) (2018)
- [Sparsified SGD with Memory](https://mlsanity.com/paper/1809.07599/) (2018)
- [TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learning](https://mlsanity.com/paper/1705.07878/) (2017)
- [PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimization](https://mlsanity.com/paper/1905.13727/) (2019)
- [Effects of Spectral Normalization in Multi-agent Reinforcement Learning](https://mlsanity.com/paper/2212.05331/) (2022)
- [Adaptive Consensus Optimization Method for GANs](https://mlsanity.com/paper/2304.10317/) (2023)
- [Angle based dynamic learning rate for gradient descent](https://mlsanity.com/paper/2304.10457/) (2023)
- [Light-weight Deep Extreme Multilabel Classification](https://mlsanity.com/paper/2304.11045/) (2023)
- [On the Byzantine Fault Tolerance of signSGD with Majority Vote](https://mlsanity.com/paper/2502.19170/) (2025)
- [A Gauss-Newton Approach for Min-Max Optimization in Generative Adversarial Networks](https://mlsanity.com/paper/2404.07172/) (2024)

## Built on by (most cited first, 2 of 2)

- [Sign compression for Muon: SignMuon, MuonSign, and the Limits of Error Feedback](https://mlsanity.com/paper/2607.29674/) (2026)
- [TACO: Ternary Absolute-max Column-wise One-sparse Optimizer for LLM Fine-Tuning](https://mlsanity.com/paper/2610.02199/) (2026)

## Abstract (arXiv)

Distributed training of large neural networks is bottlenecked by full-precision gradient communication and by coordinatewise optimizers that ignore the matrix structure of weight tensors. We propose Sign-Muon, a 1-bit, matrix-aware optimizer that combines majority-vote sign aggregation from signSGD with the polar-step framework of Muon. Each worker forms a Muon-style direction by taking the polar factor of its momentum via a Newton--Schulz iteration, transmits only the entrywise signs, and aggregates by majority vote; an optional local polar step further enforces orthogonality at no extra communication cost. Under spectral-norm smoothness and bounded-variance stochastic gradients, the spectral-norm normalized sign step yields an O(1/√T) nonconvex rate for an ℓ1-based stationarity measure. With unimodal symmetric noise, majority vote across M workers cuts the stochastic term by 1/√M, matching signSGD. In the α-β model, distributed Sign-Muon needs only one integer sum-allreduce per iteration; all orthogonalization is local, giving a 32× bandwidth reduction over float32 (4× for int8). Across 330 CIFAR-10/ResNet-50 configurations Sign-Muon attains the best validation accuracy (92.15%); its 4-GPU majority-vote variant reaches 92.02% with 37% less training time at matched effective batch. On nanoGPT, Sign-Muon achieves lower perplexity and better anytime performance than other sign-based baselines, with favorable weak-scaling up to 16 GPUs.