{"list":"other","total":51,"rows":[{"id":"1706.03762","title":"Attention Is All You Need","created":"2017-06-12","cited_by_count":60361},{"id":"1910.10683","title":"Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer","created":"2019-10-23","cited_by_count":14599},{"id":"2001.08361","title":"Scaling Laws for Neural Language Models","created":"2020-01-23","cited_by_count":6586},{"id":"2203.15556","title":"Training Compute-Optimal Large Language Models","created":"2022-03-29","cited_by_count":3019},{"id":"1804.04235","title":"Adafactor: Adaptive Learning Rates with Sublinear Memory Cost","created":"2018-04-11","cited_by_count":905},{"id":"1904.09237","title":"On the Convergence of Adam and Beyond","created":"2019-04-19","cited_by_count":841},{"id":"1503.05671","title":"Optimizing Neural Networks with Kronecker-factored Approximate Curvature","created":"2015-03-19","cited_by_count":796},{"id":"1904.00962","title":"Large Batch Optimization for Deep Learning: Training BERT in 76 minutes","created":"2019-04-01","cited_by_count":674},{"id":"1712.00409","title":"Deep Learning Scaling is Predictable, Empirically","created":"2017-12-01","cited_by_count":582},{"id":"1802.04434","title":"signSGD: Compressed Optimisation for Non-Convex Problems","created":"2018-02-13","cited_by_count":571},{"id":"2302.06675","title":"Symbolic Discovery of Optimization Algorithms","created":"2023-02-13","cited_by_count":434},{"id":"2502.16982","title":"Muon is Scalable for LLM Training","created":"2025-02-24","cited_by_count":428},{"id":"1802.09568","title":"Shampoo: Preconditioned Stochastic Tensor Optimization","created":"2018-02-26","cited_by_count":426},{"id":"2102.06701","title":"Explaining Neural Scaling Laws","created":"2021-02-12","cited_by_count":363},{"id":"2203.03466","title":"Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer","created":"2022-03-07","cited_by_count":288},{"id":"2409.20325","title":"Old Optimizer, New Norm: An Anthology","created":"2024-09-30","cited_by_count":193},{"id":"2409.11321","title":"SOAP: Improving and Stabilizing Shampoo using Adam","created":"2024-09-17","cited_by_count":192},{"id":"2502.07529","title":"Training Deep Learning Models with Norm-Constrained LMOs","created":"2025-02-11","cited_by_count":192},{"id":"1806.01811","title":"AdaGrad stepsizes: Sharp convergence over nonconvex landscapes","created":"2018-06-05","cited_by_count":182},{"id":"1705.07774","title":"Dissecting Adam: The Sign, Magnitude and Variance of Stochastic Gradients","created":"2017-05-22","cited_by_count":139},{"id":"2310.17813","title":"A Spectral Condition for Feature Learning","created":"2023-10-26","cited_by_count":109},{"id":"2503.12645","title":"Understanding Gradient Orthogonalization for Deep Learning via Non-Euclidean Trust-Region Optimization","created":"2025-03-16","cited_by_count":92},{"id":"2506.15054","title":"Muon Optimizes Under Spectral Norm Constraints","created":"2025-06-18","cited_by_count":86},{"id":"2502.02900","title":"A Note on the Convergence of Muon","created":"2025-02-05","cited_by_count":83},{"id":"2410.21265","title":"Modular Duality in Deep Learning","created":"2024-10-28","cited_by_count":77},{"id":"2505.13416","title":"Gluon: Making Muon & Scion Great Again! (Bridging Theory and Practice of LMO-based Optimizers for LLMs)","created":"2025-05-19","cited_by_count":72},{"id":"2405.14813","title":"Scalable Optimization in the Modular Norm","created":"2024-05-23","cited_by_count":68},{"id":"2503.20762","title":"ASGO: Adaptive Structured Gradient Optimization","created":"2025-03-26","cited_by_count":57},{"id":"2310.05898","title":"Lion Secretly Solves Constrained Optimization: As Lyapunov Predicts","created":"2023-10-09","cited_by_count":51},{"id":"2002.03432","title":"On the distance between two neural networks and the stability of learning","created":"2020-02-09","cited_by_count":46},{"id":"2509.26030","title":"Muon Outperforms Adam in Tail-End Associative Memory Learning","created":"2025-09-30","cited_by_count":46},{"id":"2503.10537","title":"Structured Preconditioners in Adaptive Optimization: A Unified Analysis","created":"2025-03-13","cited_by_count":33},{"id":"2512.04299","title":"When do spectral gradient updates help in deep learning?","created":"2025-12-03","cited_by_count":31},{"id":"2601.13474","title":"Preconditioning Benefits of Spectral Orthogonalization in Muon","created":"2026-01-20","cited_by_count":30},{"id":"2510.09827","title":"An Exploration of Non-Euclidean Gradient Descent: Muon and its Many Variants","created":"2025-10-10","cited_by_count":28},{"id":"2202.07052","title":"Orthogonalising gradients to speed up neural network optimisation","created":"2022-02-14","cited_by_count":27},{"id":"2402.03496","title":"Can We Remove the Square-Root in Adaptive Gradient Methods? A Second-Order Perspective","created":"2024-02-05","cited_by_count":25},{"id":"2106.02925","title":"Tensor Normal Training for Deep Learning Models","created":"2021-06-05","cited_by_count":24},{"id":"2412.13148","title":"SWAN: SGD with Normalization and Whitening Enables Stateless LLM Training","created":"2024-12-17","cited_by_count":20},{"id":"2402.04553","title":"Curvature-Informed SGD via General Purpose Lie-Group Preconditioners","created":"2024-02-07","cited_by_count":16},{"id":"2506.16659","title":"Memory-Efficient LLM Pretraining via Minimalist Optimizer Design","created":"2025-06-20","cited_by_count":13},{"id":"2510.10777","title":"Preconditioned Norms: A Unified Framework for Steepest Descent, Quasi-Newton and Adaptive Methods","created":"2025-10-12","cited_by_count":11},{"id":"2510.21800","title":"MARS-M: When Variance Reduction Meets Matrices","created":"2025-10-20","cited_by_count":10},{"id":"2502.06742","title":"Gradient Multi-Normalization for Stateless and Scalable LLM Training","created":"2025-02-10","cited_by_count":9},{"id":"1708.00523","title":"The duality structure gradient descent algorithm: analysis and applications to neural networks","created":"2017-08-01","cited_by_count":8},{"id":"2505.24275","title":"GradPower: Powering Gradients for Faster Language Model Pre-Training","created":"2025-05-30","cited_by_count":8},{"id":"1902.02257","title":"Dual Space Preconditioning for Gradient Descent","created":"2019-02-06","cited_by_count":7},{"id":"2512.22768","title":"Understanding the Mechanisms of Fast Hyperparameter Transfer","created":"2025-12-28","cited_by_count":7},{"id":"2509.24218","title":"Conda: Column-Normalized Adam for Training Large Language Models Faster","created":"2025-09-29","cited_by_count":5},{"id":"2602.13413","title":"Why is Normalization Preferred? A Worst-Case Complexity Theory for Stochastically Preconditioned SGD under Heavy-Tailed Noise","created":"2026-02-13","cited_by_count":2},{"id":"2602.10545","title":"$\\mu$pscaling small models: Principled warm starts and hyperparameter transfer","created":"2026-02-11","cited_by_count":1}],"next_cursor":null}