{"list":"other","total":59,"rows":[{"id":"1910.10683","title":"Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer","created":"2019-10-23","cited_by_count":14599},{"id":"2307.09288","title":"Llama 2: Open Foundation and Fine-Tuned Chat Models","created":"2023-07-18","cited_by_count":13028},{"id":"2406.17557","title":"The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale","created":"2024-06-25","cited_by_count":1160},{"id":"2606.19348","title":"DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence","created":"2026-04-26","cited_by_count":1040},{"id":"1804.04235","title":"Adafactor: Adaptive Learning Rates with Sublinear Memory Cost","created":"2018-04-11","cited_by_count":905},{"id":"1503.05671","title":"Optimizing Neural Networks with Kronecker-factored Approximate Curvature","created":"2015-03-19","cited_by_count":796},{"id":"2502.16982","title":"Muon is Scalable for LLM Training","created":"2025-02-24","cited_by_count":428},{"id":"1802.09568","title":"Shampoo: Preconditioned Stochastic Tensor Optimization","created":"2018-02-26","cited_by_count":426},{"id":"1905.10018","title":"Momentum-Based Variance Reduction in Non-Convex SGD","created":"2019-05-24","cited_by_count":306},{"id":"1807.01695","title":"SPIDER: Near-Optimal Non-Convex Optimization via Stochastic Path Integrated Differential Estimator","created":"2018-07-04","cited_by_count":287},{"id":"2409.20325","title":"Old Optimizer, New Norm: An Anthology","created":"2024-09-30","cited_by_count":193},{"id":"2409.11321","title":"SOAP: Improving and Stabilizing Shampoo using Adam","created":"2024-09-17","cited_by_count":192},{"id":"2502.07529","title":"Training Deep Learning Models with Norm-Constrained LMOs","created":"2025-02-11","cited_by_count":192},{"id":"2208.06677","title":"Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Models","created":"2022-08-13","cited_by_count":174},{"id":"2002.03305","title":"Momentum Improves Normalized SGD","created":"2020-02-09","cited_by_count":139},{"id":"2402.16788","title":"Why Transformers Need Adam: A Hessian Perspective","created":"2024-02-26","cited_by_count":124},{"id":"2505.16932","title":"The Polar Express: Optimal Matrix Sign Methods and Their Application to the Muon Algorithm","created":"2025-05-22","cited_by_count":110},{"id":"2304.13972","title":"Convergence of Adam Under Relaxed Assumptions","created":"2023-04-27","cited_by_count":99},{"id":"2506.15054","title":"Muon Optimizes Under Spectral Norm Constraints","created":"2025-06-18","cited_by_count":86},{"id":"2502.02900","title":"A Note on the Convergence of Muon","created":"2025-02-05","cited_by_count":83},{"id":"2505.02222","title":"Practical Efficiency of Muon for Pretraining","created":"2025-05-04","cited_by_count":80},{"id":"2309.06497","title":"A Distributed Data-Parallel PyTorch Implementation of the Distributed Shampoo Optimizer for Training Neural Networks At-Scale","created":"2023-09-12","cited_by_count":77},{"id":"2504.05295","title":"Dion: Distributed Orthonormalized Updates","created":"2025-04-07","cited_by_count":68},{"id":"2507.11005","title":"AdaMuon: Adaptive Muon Optimizer","created":"2025-07-15","cited_by_count":67},{"id":"2411.10438","title":"MARS: Unleashing the Power of Variance Reduction for Training Large Models","created":"2024-11-15","cited_by_count":64},{"id":"1806.07811","title":"Stochastic Nested Variance Reduction for Nonconvex Optimization","created":"2018-06-20","cited_by_count":63},{"id":"2503.20762","title":"ASGO: Adaptive Structured Gradient Optimization","created":"2025-03-26","cited_by_count":57},{"id":"2505.21799","title":"PolarGrad: A Class of Matrix-Gradient Optimizers from a Unifying Preconditioning Perspective","created":"2025-05-27","cited_by_count":52},{"id":"2311.00636","title":"Kronecker-Factored Approximate Curvature for Modern Neural Network Architectures","created":"2023-11-01","cited_by_count":49},{"id":"2506.04192","title":"Lions and Muons: Optimization via Stochastic Frank-Wolfe under Heavy-Tailed Noise","created":"2025-06-04","cited_by_count":41},{"id":"2106.08208","title":"SUPER-ADAM: Faster and Universal Framework of Adaptive Gradients","created":"2021-06-15","cited_by_count":40},{"id":"2310.17998","title":"Closing the Gap Between the Upper Bound and the Lower Bound of Adam's Iteration Complexity","created":"2023-10-27","cited_by_count":38},{"id":"2601.19156","title":"Convergence of Muon with Newton-Schulz","created":"2026-01-27","cited_by_count":36},{"id":"2510.16981","title":"MuonBP: Faster Muon via Block-Periodic Orthogonalization","created":"2025-10-19","cited_by_count":35},{"id":"2509.15816","title":"On the Convergence of Muon and Beyond","created":"2025-09-19","cited_by_count":31},{"id":"2507.01598","title":"Convergence Bound and Critical Batch Size of Muon Optimizer","created":"2025-07-02","cited_by_count":28},{"id":"2510.19933","title":"Beyond the Ideal: Analyzing the Inexact Muon Update","created":"2025-10-22","cited_by_count":27},{"id":"2509.02981","title":"AdaGrad Meets Muon: Adaptive Stepsizes for Orthogonal Updates","created":"2025-09-03","cited_by_count":23},{"id":"2412.13148","title":"SWAN: SGD with Normalization and Whitening Enables Stateless LLM Training","created":"2024-12-17","cited_by_count":20},{"id":"2506.07254","title":"A Stable Whitening Optimizer for Efficient Neural Network Training","created":"2025-06-08","cited_by_count":18},{"id":"2302.03764","title":"Sketchy: Memory-efficient Adaptive Regularization with Frequent Directions","created":"2023-02-07","cited_by_count":17},{"id":"2603.28254","title":"MuonEq: Balancing Before Orthogonalization with Lightweight Equilibration","created":"2026-03-30","cited_by_count":17},{"id":"2603.20527","title":"RMNP: Row-Momentum Normalized Preconditioning for Scalable Matrix-Based Optimization","created":"2026-03-20","cited_by_count":15},{"id":"2602.17080","title":"Adam Improves Muon: Adaptive Moment Estimation with Orthogonalized Momentum","created":"2026-02-19","cited_by_count":14},{"id":"2605.11181","title":"Muon is Not That Special: Random or Inverted Spectra Work Just as Well","created":"2026-05-11","cited_by_count":14},{"id":"2505.02809","title":"Towards Quantifying the Hessian Structure of Neural Networks","created":"2025-05-05","cited_by_count":13},{"id":"2506.14562","title":"AlphaDecay: Module-wise Weight Decay for Heavy-Tailed Balancing in LLMs","created":"2025-06-17","cited_by_count":13},{"id":"2603.09952","title":"On the Width Scaling of Neural Optimizers Under Matrix Operator Norms I: Row/Column Normalization and Hyperparameter Transfer","created":"2026-03-10","cited_by_count":12},{"id":"2509.23106","title":"Effective Quantization of Muon Optimizer States","created":"2025-09-27","cited_by_count":11},{"id":"2510.00643","title":"Error Feedback for Muon and Friends","created":"2025-10-01","cited_by_count":11},{"id":"2007.13985","title":"Stochastic Normalized Gradient Descent with Momentum for Large-Batch Training","created":"2020-07-28","cited_by_count":10},{"id":"2601.19400","title":"Improved Convergence Rates of Muon Optimizer for Nonconvex Optimization","created":"2026-01-27","cited_by_count":9},{"id":"2502.00213","title":"Gradient Heterogeneity Complements Hessian Heterogeneity in Transformer Optimization","created":"2025-01-31","cited_by_count":7},{"id":"2605.13079","title":"Spectral Flattening Is All Muon Needs: How Orthogonalization Controls Learning Rate and Convergence","created":"2026-05-13","cited_by_count":5},{"id":"2605.22297","title":"One LR Doesn't Fit All: Heavy-Tail Guided Layerwise Learning Rates for LLMs","created":"2026-05-21","cited_by_count":3},{"id":"2606.14560","title":"Free Heavy-Tailed Lunch for Muon: A Theoretical Justification of Empirical Success","created":"2026-06-12","cited_by_count":2},{"id":"2606.17526","title":"MGUP: A Momentum-Gradient Alignment Update Policy for Stochastic Optimization","created":"2026-06-16","cited_by_count":2},{"id":"2512.07112","title":"FOAM: Blocked State Folding for Memory-Efficient LLM Training","created":"2025-12-08","cited_by_count":1},{"id":"2606.01720","title":"A Note on Stability for Orthogonalized Matrix Momentum with Client Sampling","created":"2026-06-01","cited_by_count":1}],"next_cursor":null}