# Training Neural Networks at Any Scale

arXiv: https://arxiv.org/abs/2511.11163 · HTML: https://mlsanity.com/paper/2511.11163/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Nov 2025.

## Built on by (most cited first, 2 of 2)

- [Ky Fan Norms and Beyond: Dual Norms and Combinations for Matrix Optimization](https://mlsanity.com/paper/2512.09678/) (2025)
- [From Spectra to Joint Schedules in LLM Pre-training: 3+3(+2) Scaling-Law Regimes](https://mlsanity.com/paper/2609.40148/) (2026)

## Abstract (arXiv)

This article reviews modern optimization methods for training neural networks with an emphasis on efficiency and scale. We present state-of-the-art optimization algorithms under a unified algorithmic template that highlights the importance of adapting to the structures in the problem. We then cover how to make these algorithms agnostic to the scale of the problem. Our exposition is intended as an introduction for both practitioners and researchers who wish to be involved in these exciting new developments.