{"list":"other","total":59,"rows":[{"id":"2407.21783","title":"The Llama 3 Herd of Models","created":"2024-07-31","cited_by_count":14715},{"id":"2009.03300","title":"Measuring Massive Multitask Language Understanding","created":"2020-09-07","cited_by_count":7963},{"id":"1803.05457","title":"Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge","created":"2018-03-14","cited_by_count":4886},{"id":"1905.07830","title":"HellaSwag: Can a Machine Really Finish Your Sentence?","created":"2019-05-19","cited_by_count":4100},{"id":"2203.15556","title":"Training Compute-Optimal Large Language Models","created":"2022-03-29","cited_by_count":3019},{"id":"1809.02789","title":"Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering","created":"2018-09-08","cited_by_count":2164},{"id":"2002.05202","title":"GLU Variants Improve Transformer","created":"2020-02-12","cited_by_count":1804},{"id":"1802.05957","title":"Spectral Normalization for Generative Adversarial Networks","created":"2018-02-16","cited_by_count":1636},{"id":"1910.07467","title":"Root Mean Square Layer Normalization","created":"2019-10-16","cited_by_count":1480},{"id":"1908.03265","title":"On the Variance of the Adaptive Learning Rate and Beyond","created":"2019-08-08","cited_by_count":772},{"id":"1707.06209","title":"Crowdsourcing Multiple Choice Science Questions","created":"2017-07-19","cited_by_count":765},{"id":"2008.02275","title":"Aligning AI With Shared Human Values","created":"2020-08-05","cited_by_count":723},{"id":"1706.08498","title":"Spectrally-normalized margin bounds for neural networks","created":"2017-06-26","cited_by_count":686},{"id":"1904.00962","title":"Large Batch Optimization for Deep Learning: Training BERT in 76 minutes","created":"2019-04-01","cited_by_count":674},{"id":"1802.04434","title":"signSGD: Compressed Optimisation for Non-Convex Problems","created":"2018-02-13","cited_by_count":571},{"id":"2502.16982","title":"Muon is Scalable for LLM Training","created":"2025-02-24","cited_by_count":428},{"id":"1905.11881","title":"Why gradient clipping accelerates training: A theoretical justification for adaptivity","created":"2019-05-28","cited_by_count":341},{"id":"1905.10018","title":"Momentum-Based Variance Reduction in Non-Convex SGD","created":"2019-05-24","cited_by_count":306},{"id":"1608.05859","title":"Using the Output Embedding to Improve Language Models","created":"2016-08-20","cited_by_count":272},{"id":"1901.06053","title":"A Tail-Index Analysis of Stochastic Gradient Noise in Deep Neural Networks","created":"2019-01-18","cited_by_count":214},{"id":"2409.20325","title":"Old Optimizer, New Norm: An Anthology","created":"2024-09-30","cited_by_count":193},{"id":"2409.11321","title":"SOAP: Improving and Stabilizing Shampoo using Adam","created":"2024-09-17","cited_by_count":192},{"id":"2502.07529","title":"Training Deep Learning Models with Norm-Constrained LMOs","created":"2025-02-11","cited_by_count":192},{"id":"2309.14322","title":"Small-scale proxies for large-scale Transformer training instabilities","created":"2023-09-25","cited_by_count":183},{"id":"2405.18392","title":"Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations","created":"2024-05-28","cited_by_count":170},{"id":"2012.10333","title":"Learning from History for Byzantine Robust Optimization","created":"2020-12-18","cited_by_count":137},{"id":"2505.16932","title":"The Polar Express: Optimal Matrix Sign Methods and Their Application to the Muon Algorithm","created":"2025-05-22","cited_by_count":110},{"id":"2005.10785","title":"Stochastic Optimization with Heavy-Tailed Noise via Accelerated Gradient Clipping","created":"2020-05-21","cited_by_count":104},{"id":"2305.01588","title":"Revisiting Gradient Clipping: Stochastic bias and tight convergence guarantees","created":"2023-05-02","cited_by_count":83},{"id":"1803.08917","title":"Byzantine Stochastic Gradient Descent","created":"2018-03-23","cited_by_count":81},{"id":"2505.13416","title":"Gluon: Making Muon & Scion Great Again! (Bridging Theory and Practice of LMO-based Optimizers for LLMs)","created":"2025-05-19","cited_by_count":72},{"id":"2509.01440","title":"Benchmarking Optimizers for Large Language Model Pretraining","created":"2025-09-01","cited_by_count":68},{"id":"2507.11005","title":"AdaMuon: Adaptive Muon Optimizer","created":"2025-07-15","cited_by_count":67},{"id":"2411.10438","title":"MARS: Unleashing the Power of Variance Reduction for Training Large Models","created":"2024-11-15","cited_by_count":64},{"id":"2503.20762","title":"ASGO: Adaptive Structured Gradient Optimization","created":"2025-03-26","cited_by_count":57},{"id":"2409.03137","title":"The AdEMAMix Optimizer: Better, Faster, Older","created":"2024-09-05","cited_by_count":54},{"id":"2306.00204","title":"Toward Understanding Why Adam Converges Faster Than SGD for Transformers","created":"2023-05-31","cited_by_count":53},{"id":"1804.09554","title":"Stochastic Conditional Gradient Methods: From Convex Minimization to Submodular Maximization","created":"2018-04-24","cited_by_count":47},{"id":"2503.10537","title":"Structured Preconditioners in Adaptive Optimization: A Unified Analysis","created":"2025-03-13","cited_by_count":33},{"id":"2512.04299","title":"When do spectral gradient updates help in deep learning?","created":"2025-12-03","cited_by_count":31},{"id":"2501.06842","title":"SPAM: Spike-Aware Adam with Momentum Reset for Stable LLM Training","created":"2025-01-12","cited_by_count":28},{"id":"2510.09827","title":"An Exploration of Non-Euclidean Gradient Descent: Muon and its Many Variants","created":"2025-10-10","cited_by_count":28},{"id":"2409.14989","title":"Methods for Convex $(L_0,L_1)$-Smooth Optimization: Clipping, Acceleration, and Adaptivity","created":"2024-09-23","cited_by_count":25},{"id":"2410.08198","title":"Adam Exploits $\\ell_\\infty$-geometry of Loss Landscape via Coordinate-wise Adaptivity","created":"2024-10-10","cited_by_count":23},{"id":"2509.02981","title":"AdaGrad Meets Muon: Adaptive Stepsizes for Orthogonal Updates","created":"2025-09-03","cited_by_count":23},{"id":"2406.04443","title":"Clipping Improves Adam-Norm and AdaGrad-Norm when the Noise Is Heavy-Tailed","created":"2024-06-06","cited_by_count":18},{"id":"1804.08544","title":"A Conditional Gradient Framework for Composite Convex Minimization with Applications to Semidefinite Programming","created":"2018-04-23","cited_by_count":14},{"id":"2201.08663","title":"Fast Differentiable Matrix Square Root","created":"2022-01-21","cited_by_count":14},{"id":"1302.2325","title":"Conditional Gradient Algorithms for Norm-Regularized Smooth Convex Optimization","created":"2013-02-10","cited_by_count":13},{"id":"2502.07923","title":"Sign Operator for Coping with Heavy-Tailed Noise in Non-Convex Optimization: High Probability Bounds Under $(L_0, L_1)$-Smoothness","created":"2025-02-11","cited_by_count":13},{"id":"2602.11948","title":"Insights on Muon from Simple Quadratics","created":"2026-02-12","cited_by_count":13},{"id":"2402.16017","title":"Spectrum Extraction and Clipping for Implicitly Linear Layers","created":"2024-02-25","cited_by_count":7},{"id":"2502.00213","title":"Gradient Heterogeneity Complements Hessian Heterogeneity in Transformer Optimization","created":"2025-01-31","cited_by_count":7},{"id":"2510.02239","title":"Drop-Muon: Update Less, Converge Faster","created":"2025-10-02","cited_by_count":7},{"id":"2311.14127","title":"Byzantine Robustness and Partial Participation Can Be Achieved at Once: Just Clip Gradient Differences","created":"2023-11-23","cited_by_count":5},{"id":"2509.24218","title":"Conda: Column-Normalized Adam for Training Large Language Models Faster","created":"2025-09-29","cited_by_count":5},{"id":"2512.00763","title":"Provable Benefit of SignGD: A Minimal Model Under Heavy-Tailed Class Imbalance","created":"2025-11-30","cited_by_count":5},{"id":"2412.01168","title":"On the Surprising Effectiveness of Spectral Clipping in Learning Stable Linear and Latent-Linear Dynamical Systems","created":"2024-12-02","cited_by_count":1},{"id":"2512.05327","title":"Non-Convex Federated Optimization under Cost-Aware Client Selection","created":"2025-12-05","cited_by_count":1}],"next_cursor":null}