{"total":80,"rows":[{"id":"2406.16793","title":"Adam-mini: Use Fewer Learning Rates To Gain More","created":"2024-06-24","cited_by_count":137,"area":null},{"id":"2509.26030","title":"Muon Outperforms Adam in Tail-End Associative Memory Learning","created":"2025-09-30","cited_by_count":46,"area":null},{"id":"2407.07972","title":"Deconstructing What Makes a Good Optimizer for Language Models","created":"2024-07-10","cited_by_count":41,"area":null},{"id":"2507.07101","title":"Small Batch Size Training for Language Models: When Vanilla SGD Works, and Why Gradient Accumulation Is Wasteful","created":"2025-07-09","cited_by_count":41,"area":null},{"id":"2505.21829","title":"In Search of Adam's Secret Sauce","created":"2025-05-27","cited_by_count":40,"area":null},{"id":"2511.00674","title":"Isotropic Curvature Model for Understanding Deep Learning Optimization: Is Gradient Orthogonalization Optimal?","created":"2025-11-01","cited_by_count":37,"area":null},{"id":"2405.16002","title":"Does SGD really happen in tiny subspaces?","created":"2024-05-25","cited_by_count":36,"area":null},{"id":"2502.15015","title":"Accelerating Neural Network Training: An Analysis of the AlgoPerf Competition","created":"2025-02-20","cited_by_count":32,"area":null},{"id":"2502.19002","title":"The Sharpness Disparity Principle in Transformers for Accelerating Language Model Pre-Training","created":"2025-02-26","cited_by_count":30,"area":null},{"id":"2505.19227","title":"Scaling Laws for Gradient Descent and Sign Descent for Linear Bigram Models under Zipf's Law","created":"2025-05-25","cited_by_count":24,"area":null},{"id":"2405.18199","title":"Adam with model exponential moving average is effective for nonconvex optimization","created":"2024-05-28","cited_by_count":23,"area":null},{"id":"2410.08198","title":"Adam Exploits $\\ell_\\infty$-geometry of Loss Landscape via Coordinate-wise Adaptivity","created":"2024-10-10","cited_by_count":23,"area":null},{"id":"2412.13148","title":"SWAN: SGD with Normalization and Whitening Enables Stateless LLM Training","created":"2024-12-17","cited_by_count":20,"area":null},{"id":"2510.22980","title":"How Muon's Spectral Design Benefits Generalization: A Study on Imbalanced Data","created":"2025-10-27","cited_by_count":19,"area":null},{"id":"2410.16561","title":"Revisiting Gradient Normalization and Clipping for Nonconvex SGD under Heavy-Tailed Noise: Necessity, Sufficiency, and Acceleration","created":"2024-10-21","cited_by_count":18,"area":null},{"id":"2506.12543","title":"Is your batch size the problem? Revisiting the Adam-SGD gap in language modeling","created":"2025-06-14","cited_by_count":15,"area":null},{"id":"2405.21070","title":"What Makes CLIP More Robust to Long-Tailed Pre-Training Data? A Controlled Study for Transferable Insights","created":"2024-05-31","cited_by_count":13,"area":null},{"id":"2505.02809","title":"Towards Quantifying the Hessian Structure of Neural Networks","created":"2025-05-05","cited_by_count":13,"area":null},{"id":"2505.24022","title":"The Rich and the Simple: On the Implicit Bias of Adam and SGD","created":"2025-05-29","cited_by_count":13,"area":null},{"id":"2506.16659","title":"Memory-Efficient LLM Pretraining via Minimalist Optimizer Design","created":"2025-06-20","cited_by_count":13,"area":null},{"id":"2601.15892","title":"Stable-DiffCoder: Pushing the Frontier of Code Diffusion Large Language Model","created":"2026-01-22","cited_by_count":13,"area":null},{"id":"2601.23000","title":"Mano: Restriking Manifold Optimization for LLM Training","created":"2026-01-30","cited_by_count":12,"area":null},{"id":"2410.19964","title":"Understanding Adam Requires Better Rotation Dependent Assumptions","created":"2024-10-25","cited_by_count":11,"area":null},{"id":"2605.09253","title":"Cornerstones or Stumbling Blocks? Deciphering the Rock Tokens in On-Policy Distillation","created":"2026-05-10","cited_by_count":11,"area":"distill"},{"id":"2405.19279","title":"Understanding and Minimising Outlier Features in Neural Network Training","created":"2024-05-29","cited_by_count":10,"area":null},{"id":"2502.06742","title":"Gradient Multi-Normalization for Stateless and Scalable LLM Training","created":"2025-02-10","cited_by_count":9,"area":null},{"id":"2505.24275","title":"GradPower: Powering Gradients for Faster Language Model Pre-Training","created":"2025-05-30","cited_by_count":8,"area":null},{"id":"2602.09314","title":"Clarifying Shampoo: Adapting Spectral Descent to Stochasticity and the Parameter Trajectory","created":"2026-02-10","cited_by_count":8,"area":null},{"id":"2411.00999","title":"Normalization Layer Per-Example Gradients are Sufficient to Predict Gradient Noise Scale in Transformers","created":"2024-11-01","cited_by_count":7,"area":null},{"id":"2502.04164","title":"Efficient Distributed Optimization under Heavy-Tailed Noise","created":"2025-02-06","cited_by_count":7,"area":null},{"id":"2510.03817","title":"TROLL: Trust Regions improve Reinforcement Learning for Large Language Models","created":"2025-10-04","cited_by_count":7,"area":null},{"id":"2510.03866","title":"On Provable Benefits of Muon in Federated Learning","created":"2025-10-04","cited_by_count":7,"area":null},{"id":"2510.19376","title":"Optimization Benchmark for Diffusion Models on Dynamical Systems","created":"2025-10-22","cited_by_count":7,"area":"diffusion"},{"id":"2602.15322","title":"On Surprising Effectiveness of Masking Updates in Adaptive Optimizers","created":"2026-02-17","cited_by_count":7,"area":null},{"id":"2606.04662","title":"Why Muon Outperforms Adam: A Curvature Perspective","created":"2026-06-03","cited_by_count":6,"area":null},{"id":"2502.17009","title":"Unbiased and Sign Compression in Distributed Learning: Comparing Noise Resilience via SDEs","created":"2025-02-24","cited_by_count":5,"area":null},{"id":"2512.00763","title":"Provable Benefit of SignGD: A Minimal Model Under Heavy-Tailed Class Imbalance","created":"2025-11-30","cited_by_count":5,"area":null},{"id":"2412.21124","title":"Adaptive Batch Size Schedules for Distributed Training of Language Models with Data and Model Parallelism","created":"2024-12-30","cited_by_count":4,"area":null},{"id":"2506.00181","title":"On the Interaction of Batch Noise, Adaptivity, and Compression, under $(L_0,L_1)$-Smoothness: An SDE Approach","created":"2025-05-30","cited_by_count":4,"area":null},{"id":"2507.05966","title":"Simple Convergence Proof of Adam From a Sign-like Descent Perspective","created":"2025-07-08","cited_by_count":4,"area":null},{"id":"2510.26997","title":"Gradient Descent as Loss Landscape Navigation: a Normative Framework for Deriving Learning Rules","created":"2025-10-30","cited_by_count":4,"area":null},{"id":"2603.09581","title":"Towards Understanding Adam Convergence on Highly Degenerate Polynomials","created":"2026-03-10","cited_by_count":4,"area":null},{"id":"2503.01843","title":"When Can You Get Away with Low Memory Adam?","created":"2025-03-03","cited_by_count":3,"area":null},{"id":"2507.08963","title":"Stochastic Approximation with Block Coordinate Optimal Stepsizes","created":"2025-07-11","cited_by_count":3,"area":null},{"id":"2606.09658","title":"Muon Learns More Robust and Transferable Features than Adam","created":"2026-06-08","cited_by_count":3,"area":null},{"id":"2510.26303","title":"Implicit Bias of Per-sample Adam on Separable Data: Departure from the Full-batch Regime","created":"2025-10-30","cited_by_count":2,"area":null},{"id":"2511.04456","title":"Federated Stochastic Minimax Optimization under Heavy-Tailed Noises","created":"2025-11-06","cited_by_count":2,"area":null},{"id":"2602.03515","title":"Mitigating Staleness in Asynchronous Pipeline Parallelism via Basis Rotation","created":"2026-02-03","cited_by_count":2,"area":null},{"id":"2605.17787","title":"Revisiting the Adam-SGD Gap in LLM Pre-Training: The Role of Large Effective Learning Rates","created":"2026-05-18","cited_by_count":2,"area":null},{"id":"2605.18528","title":"Tight Complexity of Muon under Heavy-Tailed Noise","created":"2026-05-18","cited_by_count":2,"area":null},{"id":"2412.02153","title":"Revisiting the Initial Steps in Adaptive Gradient Descent Optimization","created":"2024-12-03","cited_by_count":1,"area":null},{"id":"2501.11638","title":"Class Imbalance in Anomaly Detection: Learning from an Exactly Solvable Model","created":"2025-01-20","cited_by_count":1,"area":null},{"id":"2505.13122","title":"When majority rules, minority loses: bias amplification of gradient descent","created":"2025-05-19","cited_by_count":1,"area":null},{"id":"2505.15013","title":"Convergence of Adam in Deep ReLU Networks via Directional Complexity and Kakeya Bounds","created":"2025-05-21","cited_by_count":1,"area":null},{"id":"2505.21857","title":"Optimizable Model Averaging for Ensembling Heterogeneous Model Outputs","created":"2025-05-28","cited_by_count":1,"area":null},{"id":"2507.06464","title":"SoftSignSGD(S3): An Enhanced Optimizer for Practical DNN Training and Loss Spikes Minimization Beyond Adam","created":"2025-07-09","cited_by_count":1,"area":null},{"id":"2512.04632","title":"Turbo-Muon: Almost-Orthogonal Pre-Conditioning for Fast Muon Updates","created":"2025-12-04","cited_by_count":1,"area":null},{"id":"2602.10657","title":"Benchmarks Are Not That Out of Distribution: Word Overlap Predicts Performance","created":"2026-02-11","cited_by_count":1,"area":null},{"id":"2602.19945","title":"DP-FedAdamW: An Efficient Optimizer for Differentially Private Federated Large Models","created":"2026-02-23","cited_by_count":1,"area":null},{"id":"2605.04269","title":"Adapt or Forget: Provable Tradeoffs Between Adam and SGD in Nonstationary Optimization","created":"2026-05-05","cited_by_count":1,"area":null},{"id":"2606.14259","title":"Beyond a Single Explanation of the Adam--SGD Gap","created":"2026-06-12","cited_by_count":1,"area":null},{"id":"2606.23676","title":"Open Problem: Is AdamW Effective Under Heavy-Tailed Noise?","created":"2026-06-22","cited_by_count":1,"area":null},{"id":"2607.01455","title":"Ember: A Kilobyte-State Optimizer for Token Interfaces","created":"2026-07-01","cited_by_count":1,"area":null},{"id":"2411.16303","title":"Understanding Generalization of Federated Learning: the Trade-off between Model Stability and Optimization","created":"2024-11-25","cited_by_count":0,"area":null},{"id":"2507.10536","title":"On the Performance of Differentially Private Optimization with Heavy-Tail Class Imbalance","created":"2025-07-14","cited_by_count":0,"area":null},{"id":"2601.04563","title":"A Vision for Multisensory Intelligence: Sensing, Science, and Synergy","created":"2026-01-08","cited_by_count":0,"area":null},{"id":"2603.02620","title":"Same Error, Different Function: The Optimizer as an Implicit Prior in Financial Time Series","created":"2026-03-03","cited_by_count":0,"area":null},{"id":"2605.02143","title":"Personalized Federated Learning for Gradient Alignment","created":"2026-05-04","cited_by_count":0,"area":null},{"id":"2605.04895","title":"Regime-Conditioned Evaluation in Multi-Context Bayesian Optimization","created":"2026-05-06","cited_by_count":0,"area":null},{"id":"2605.05794","title":"Revealing Modular Gradient Noise Imbalance in LLMs: Calibrating Adam via Signal-to-Noise Ratio","created":"2026-05-07","cited_by_count":0,"area":null},{"id":"2605.06166","title":"One Algorithm, Two Goals: Dual Scoring for Parameter and Data Selection in LLM Fine-Tuning","created":"2026-05-07","cited_by_count":0,"area":null},{"id":"2605.28961","title":"Dynamics of Stochastic Momentum with Sparse Updates in High Dimensions","created":"2026-05-27","cited_by_count":0,"area":null},{"id":"2607.07573","title":"Multi-Class vs. Multi-Label BERT for CVE-to-CWE Mapping: How Taxonomy Structure Shapes the Errors","created":"2026-07-08","cited_by_count":0,"area":null},{"id":"2608.03941","title":"Muon Meets Mamba: Spectral Optimization for State Space Models","created":"2026-08-04","cited_by_count":0,"area":"ssm"},{"id":"2608.05136","title":"The Loss Does Not See the Basis, but Adam Does","created":"2026-08-05","cited_by_count":0,"area":null},{"id":"2608.16760","title":"On the Principles Behind Neural Network Optimizers","created":"2026-08-17","cited_by_count":0,"area":null},{"id":"2609.32593","title":"Age of Learning: Temporal Persistence of Prediction Errors as a Learning Signal","created":"2026-09-26","cited_by_count":0,"area":null},{"id":"2609.37535","title":"Why Adaptive Optimizers Underestimate Rare Tokens","created":"2026-09-29","cited_by_count":0,"area":null},{"id":"2609.37745","title":"Optimizer-dependent training dynamics converge to the same one-third optimal data scaling","created":"2026-09-29","cited_by_count":0,"area":null},{"id":"2610.19033","title":"How Many Ranks Does a Token Need? A Noise-Floor Squelch for Rank-One LoRA Experts","created":"2026-10-13","cited_by_count":0,"area":"moe"}],"next_cursor":null}