{"list":"other","total":67,"rows":[{"id":"1412.6980","title":"Adam: A Method for Stochastic Optimization","created":"2014-12-22","cited_by_count":37240},{"id":"2010.01412","title":"Sharpness-Aware Minimization for Efficiently Improving Generalization","created":"2020-10-03","cited_by_count":1417},{"id":"1609.04836","title":"On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima","created":"2016-09-15","cited_by_count":1352},{"id":"1803.05407","title":"Averaging Weights Leads to Wider Optima and Better Generalization","created":"2018-03-14","cited_by_count":1209},{"id":"2406.17557","title":"The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale","created":"2024-06-25","cited_by_count":1160},{"id":"1712.09913","title":"Visualizing the Loss Landscape of Neural Nets","created":"2017-12-28","cited_by_count":1130},{"id":"1804.04235","title":"Adafactor: Adaptive Learning Rates with Sublinear Memory Cost","created":"2018-04-11","cited_by_count":905},{"id":"1904.09237","title":"On the Convergence of Adam and Beyond","created":"2019-04-19","cited_by_count":841},{"id":"1503.05671","title":"Optimizing Neural Networks with Kronecker-factored Approximate Curvature","created":"2015-03-19","cited_by_count":796},{"id":"1908.03265","title":"On the Variance of the Adaptive Learning Rate and Beyond","created":"2019-08-08","cited_by_count":772},{"id":"1703.04933","title":"Sharp Minima Can Generalize For Deep Nets","created":"2017-03-15","cited_by_count":466},{"id":"2502.16982","title":"Muon is Scalable for LLM Training","created":"2025-02-24","cited_by_count":428},{"id":"2103.00065","title":"Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability","created":"2021-02-26","cited_by_count":427},{"id":"1802.09568","title":"Shampoo: Preconditioned Stochastic Tensor Optimization","created":"2018-02-26","cited_by_count":426},{"id":"2102.11600","title":"ASAM: Adaptive Sharpness-Aware Minimization for Scale-Invariant Learning of Deep Neural Networks","created":"2021-02-23","cited_by_count":293},{"id":"1706.04454","title":"Empirical Analysis of the Hessian of Over-Parametrized Neural Networks","created":"2017-06-14","cited_by_count":290},{"id":"2010.07468","title":"AdaBelief Optimizer: Adapting Stepsizes by the Belief in Observed Gradients","created":"2020-10-15","cited_by_count":283},{"id":"1812.04754","title":"Gradient Descent Happens in a Tiny Subspace","created":"2018-12-12","cited_by_count":217},{"id":"2409.20325","title":"Old Optimizer, New Norm: An Anthology","created":"2024-09-30","cited_by_count":193},{"id":"2409.11321","title":"SOAP: Improving and Stabilizing Shampoo using Adam","created":"2024-09-17","cited_by_count":192},{"id":"2502.07529","title":"Training Deep Learning Models with Norm-Constrained LMOs","created":"2025-02-11","cited_by_count":192},{"id":"1808.02941","title":"On the Convergence of A Class of Adam-Type Algorithms for Non-Convex Optimization","created":"2018-08-08","cited_by_count":174},{"id":"2003.02395","title":"A Simple Convergence Proof of Adam and Adagrad","created":"2020-03-05","cited_by_count":171},{"id":"1806.03884","title":"Fast Approximate Natural Gradient Descent in a Kronecker-factored Eigenbasis","created":"2018-06-11","cited_by_count":146},{"id":"2110.03141","title":"Efficient Sharpness-aware Minimization for Improved Training of Neural Networks","created":"2021-10-07","cited_by_count":137},{"id":"2406.16793","title":"Adam-mini: Use Fewer Learning Rates To Gain More","created":"2024-06-24","cited_by_count":137},{"id":"1811.09358","title":"A Sufficient Condition for Convergences of Adam and RMSProp","created":"2018-11-23","cited_by_count":126},{"id":"2402.16788","title":"Why Transformers Need Adam: A Hessian Perspective","created":"2024-02-26","cited_by_count":124},{"id":"2208.09632","title":"Adam Can Converge Without Any Modification On Update Rules","created":"2022-08-20","cited_by_count":114},{"id":"2509.02046","title":"Fantastic Pretraining Optimizers and Where to Find Them","created":"2025-09-02","cited_by_count":100},{"id":"2304.13972","title":"Convergence of Adam Under Relaxed Assumptions","created":"2023-04-27","cited_by_count":99},{"id":"2205.14083","title":"Sharpness-Aware Training for Free","created":"2022-05-27","cited_by_count":95},{"id":"2302.07011","title":"A Modern Look at the Relationship between Sharpness and Generalization","created":"2023-02-14","cited_by_count":92},{"id":"2503.12645","title":"Understanding Gradient Orthogonalization for Deep Learning via Non-Euclidean Trust-Region Optimization","created":"2025-03-16","cited_by_count":92},{"id":"1808.05671","title":"On the Convergence of Adaptive Gradient Methods for Nonconvex Optimization","created":"2018-08-16","cited_by_count":91},{"id":"2506.15054","title":"Muon Optimizes Under Spectral Norm Constraints","created":"2025-06-18","cited_by_count":86},{"id":"2510.05491","title":"NorMuon: Making Muon more efficient and scalable","created":"2025-10-07","cited_by_count":84},{"id":"2502.02900","title":"A Note on the Convergence of Muon","created":"2025-02-05","cited_by_count":83},{"id":"2402.19449","title":"Heavy-Tailed Class Imbalance and Why Adam Outperforms Gradient Descent on Language Models","created":"2024-02-29","cited_by_count":80},{"id":"2505.02222","title":"Practical Efficiency of Muon for Pretraining","created":"2025-05-04","cited_by_count":80},{"id":"2410.21265","title":"Modular Duality in Deep Learning","created":"2024-10-28","cited_by_count":77},{"id":"2504.05295","title":"Dion: Distributed Orthonormalized Updates","created":"2025-04-07","cited_by_count":68},{"id":"2507.11005","title":"AdaMuon: Adaptive Muon Optimizer","created":"2025-07-15","cited_by_count":67},{"id":"2503.20762","title":"ASGO: Adaptive Structured Gradient Optimization","created":"2025-03-26","cited_by_count":57},{"id":"2306.00204","title":"Toward Understanding Why Adam Converges Faster Than SGD for Transformers","created":"2023-05-31","cited_by_count":53},{"id":"2505.21799","title":"PolarGrad: A Class of Matrix-Gradient Optimizers from a Unifying Preconditioning Perspective","created":"2025-05-27","cited_by_count":52},{"id":"2112.03459","title":"A Novel Convergence Analysis for Algorithms of the Adam Family","created":"2021-12-07","cited_by_count":50},{"id":"1910.05929","title":"Emergent properties of the local geometry of neural loss landscapes","created":"2019-10-14","cited_by_count":46},{"id":"2509.26030","title":"Muon Outperforms Adam in Tail-End Associative Memory Learning","created":"2025-09-30","cited_by_count":46},{"id":"2307.11007","title":"Sharpness Minimization Algorithms Do Not Only Minimize Sharpness To Achieve Better Generalization","created":"2023-07-20","cited_by_count":39},{"id":"2601.19156","title":"Convergence of Muon with Newton-Schulz","created":"2026-01-27","cited_by_count":36},{"id":"2510.16981","title":"MuonBP: Faster Muon via Block-Periodic Orthogonalization","created":"2025-10-19","cited_by_count":35},{"id":"2502.19002","title":"The Sharpness Disparity Principle in Transformers for Accelerating Language Model Pre-Training","created":"2025-02-26","cited_by_count":30},{"id":"2601.13474","title":"Preconditioning Benefits of Spectral Orthogonalization in Muon","created":"2026-01-20","cited_by_count":30},{"id":"2510.19933","title":"Beyond the Ideal: Analyzing the Inexact Muon Update","created":"2025-10-22","cited_by_count":27},{"id":"2510.22980","title":"How Muon's Spectral Design Benefits Generalization: A Study on Imbalanced Data","created":"2025-10-27","cited_by_count":19},{"id":"2509.11983","title":"Low-rank Orthogonalization for Large-scale Matrix Optimization with Applications to Foundation Model Training","created":"2025-09-15","cited_by_count":18},{"id":"2604.09967","title":"Muon$^2$: Boosting Muon via Adaptive Second-Moment Preconditioning","created":"2026-04-11","cited_by_count":17},{"id":"2510.27403","title":"FedMuon: Accelerating Federated Learning with Matrix Orthogonalization","created":"2025-10-31","cited_by_count":16},{"id":"2603.26554","title":"Sharp Capacity Scaling of Spectral Optimizers in Learning Associative Memory","created":"2026-03-27","cited_by_count":14},{"id":"2505.02809","title":"Towards Quantifying the Hessian Structure of Neural Networks","created":"2025-05-05","cited_by_count":13},{"id":"2602.05725","title":"Muon in Associative Memory Learning: Training Dynamics and Scaling Laws","created":"2026-02-05","cited_by_count":13},{"id":"2310.00692","title":"A Theoretical Analysis of Noise Geometry in Stochastic Gradient Descent","created":"2023-10-01","cited_by_count":9},{"id":"2602.04669","title":"Delving into Muon and Beyond: Deep Analysis and Extensions","created":"2026-02-04","cited_by_count":9},{"id":"2601.16979","title":"A Scalable Measure of Loss Landscape Curvature for Analyzing the Training Dynamics of LLMs","created":"2026-01-23","cited_by_count":8},{"id":"2602.22681","title":"Accelerating LLM Pre-Training through Flat-Direction Dynamics Enhancement","created":"2026-02-26","cited_by_count":8},{"id":"2602.13498","title":"TrasMuon: Trust-Region Adaptive Scaling for Orthogonalized Momentum Optimizers","created":"2026-02-13","cited_by_count":6}],"next_cursor":null}