{"total":60,"rows":[{"id":"2409.03137","title":"The AdEMAMix Optimizer: Better, Faster, Older","created":"2024-09-05","cited_by_count":54,"area":null},{"id":"2502.04664","title":"Implicit Bias of Spectral Descent and Muon on Multiclass Separable Data","created":"2025-02-07","cited_by_count":41,"area":null},{"id":"2202.04295","title":"On Almost Sure Convergence Rates of Stochastic Gradient Methods","created":"2022-02-09","cited_by_count":35,"area":null},{"id":"2107.00595","title":"Fast Margin Maximization via Dual Acceleration","created":"2021-07-01","cited_by_count":33,"area":null},{"id":"2305.15155","title":"Momentum Provably Improves Error Feedback!","created":"2023-05-24","cited_by_count":31,"area":null},{"id":"2208.09900","title":"Provable Adaptivity of Adam under Non-uniform Smoothness","created":"2022-08-21","cited_by_count":30,"area":null},{"id":"2206.00457","title":"Automatic differentiation of nonsmooth iterative algorithms","created":"2022-05-31","cited_by_count":25,"area":null},{"id":"2110.03891","title":"Does Momentum Change the Implicit Regularization on Separable Data?","created":"2021-10-08","cited_by_count":22,"area":null},{"id":"2402.03982","title":"On Convergence of Adam for Stochastic Optimization under Relaxed Assumptions","created":"2024-02-06","cited_by_count":21,"area":null},{"id":"2201.11411","title":"Restarted Nonconvex Accelerated Gradient Descent: No More Polylogarithmic Factor in the $O(\\epsilon^{-7/4})$ Complexity","created":"2022-01-27","cited_by_count":17,"area":null},{"id":"2402.13794","title":"Revisiting Convergence of AdaGrad with Relaxed Assumptions","created":"2024-02-21","cited_by_count":15,"area":null},{"id":"2206.07553","title":"On the fast convergence of minibatch heavy ball momentum","created":"2022-06-15","cited_by_count":13,"area":null},{"id":"2406.04142","title":"Stochastic Polyak Step-sizes and Momentum: Convergence Guarantees and Practical Performance","created":"2024-06-06","cited_by_count":13,"area":null},{"id":"2206.11872","title":"Provable Acceleration of Heavy Ball beyond Quadratics for a Class of Polyak-\\L{}ojasiewicz Functions when the Non-Convexity is Averaged-Out","created":"2022-06-22","cited_by_count":11,"area":null},{"id":"2106.03034","title":"Minibatch and Momentum Model-based Methods for Stochastic Weakly Convex Optimization","created":"2021-06-06","cited_by_count":10,"area":null},{"id":"2403.05293","title":"Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks","created":"2024-03-08","cited_by_count":9,"area":null},{"id":"2106.02985","title":"Escaping Saddle Points Faster with Stochastic Momentum","created":"2021-06-05","cited_by_count":8,"area":null},{"id":"2110.09057","title":"Training Deep Neural Networks with Adaptive Momentum Inspired by the Quadratic Optimization","created":"2021-10-18","cited_by_count":8,"area":null},{"id":"2302.03550","title":"Exponential convergence rates for momentum stochastic gradient descent in the overparametrized setting","created":"2023-02-07","cited_by_count":8,"area":null},{"id":"2311.02000","title":"High Probability Convergence of Adam Under Unbounded Gradients and Affine Variance Noise","created":"2023-11-03","cited_by_count":8,"area":null},{"id":"2102.07314","title":"The Role of Momentum Parameters in the Optimal Convergence of Adaptive Polyak's Heavy-ball Methods","created":"2021-02-15","cited_by_count":7,"area":null},{"id":"2110.04243","title":"Heavy Ball Momentum for Conditional Gradient","created":"2021-10-08","cited_by_count":7,"area":null},{"id":"2203.15797","title":"Convergence of First-Order Methods for Constrained Nonconvex Optimization with Dependent Data","created":"2022-03-29","cited_by_count":7,"area":null},{"id":"2102.11755","title":"Analytical Study of Momentum-Based Acceleration Methods in Paradigmatic High-Dimensional Non-Convex Problems","created":"2021-02-23","cited_by_count":5,"area":null},{"id":"2205.14811","title":"Last-iterate convergence analysis of stochastic momentum methods for neural networks","created":"2022-05-30","cited_by_count":5,"area":null},{"id":"2504.01898","title":"Analysis of an Idealized Stochastic Polyak Method and its Application to Black-Box Model Distillation","created":"2025-04-02","cited_by_count":5,"area":null},{"id":"2104.14101","title":"Fast Convex Quadratic Optimization Solvers with Adaptive Sketching-based Preconditioners","created":"2021-04-29","cited_by_count":3,"area":null},{"id":"2410.16849","title":"Polyak's Heavy Ball Method Achieves Accelerated Local Rate of Convergence under Polyak-Lojasiewicz Inequality","created":"2024-10-22","cited_by_count":3,"area":null},{"id":"2507.03404","title":"On the Effectiveness of the z-Transform Method in Quadratic Optimization","created":"2025-07-04","cited_by_count":3,"area":null},{"id":"2508.00775","title":"Learning to optimize with guarantees: a complete characterization of linearly convergent algorithms","created":"2025-08-01","cited_by_count":3,"area":null},{"id":"2606.00520","title":"In-Expectation Convergence of Stochastic Gradient Methods under Heavy-Tailed Noise","created":"2026-05-30","cited_by_count":3,"area":null},{"id":"2106.02888","title":"Bandwidth-based Step-Sizes for Non-Convex Stochastic Optimization","created":"2021-06-05","cited_by_count":2,"area":null},{"id":"2111.11004","title":"Gradient Temporal Difference with Momentum: Stability and Convergence","created":"2021-11-22","cited_by_count":2,"area":null},{"id":"2303.16241","title":"Convergence of the Stochastic Heavy Ball Method With Approximate Gradients and/or Block Updating","created":"2023-03-28","cited_by_count":2,"area":null},{"id":"2403.00853","title":"Parallel Momentum Methods Under Biased Gradient Estimations","created":"2024-02-29","cited_by_count":2,"area":null},{"id":"2405.18222","title":"From Learning to Optimize to Learning Optimization Algorithms","created":"2024-05-28","cited_by_count":2,"area":null},{"id":"2406.13041","title":"Accelerated Stochastic Min-Max Optimization Based on Bias-corrected Momentum","created":"2024-06-18","cited_by_count":2,"area":null},{"id":"2506.16283","title":"Random feature approximation for general spectral methods","created":"2025-06-19","cited_by_count":2,"area":null},{"id":"2511.14501","title":"Improved Convergence in Parameter-Agnostic Error Feedback through Momentum","created":"2025-11-18","cited_by_count":2,"area":null},{"id":"2606.17526","title":"MGUP: A Momentum-Gradient Alignment Update Policy for Stochastic Optimization","created":"2026-06-16","cited_by_count":2,"area":null},{"id":"2106.06753","title":"Scaling transition from momentum stochastic gradient descent to plain stochastic gradient descent","created":"2021-06-12","cited_by_count":1,"area":null},{"id":"2110.03124","title":"Improving Adversarial Robustness for Free with Snapshot Ensemble","created":"2021-10-07","cited_by_count":1,"area":null},{"id":"2209.11944","title":"Communication-Efficient {Federated} Learning Using Censored Heavy Ball Descent","created":"2022-09-24","cited_by_count":1,"area":null},{"id":"2401.06738","title":"(Accelerated) Noise-adaptive Stochastic Heavy-Ball Momentum","created":"2024-01-12","cited_by_count":1,"area":null},{"id":"2506.11904","title":"Convergence of Momentum-Based Optimization Algorithms with Time-Varying Parameters","created":"2025-06-13","cited_by_count":1,"area":null},{"id":"2508.06743","title":"Analysis of Schedule-Free Nonconvex Optimization","created":"2025-08-08","cited_by_count":1,"area":null},{"id":"2607.08104","title":"Vanilla SGD with Momentum Survives Heavy-Tailed Noise: Convergence Analysis without Gradient Clipping or Normalization","created":"2026-07-09","cited_by_count":1,"area":null},{"id":"2102.13653","title":"On the Generalization of Stochastic Gradient Descent with Momentum","created":"2021-02-26","cited_by_count":0,"area":null},{"id":"2104.07361","title":"Scale Invariant Monte Carlo under Linear Function Approximation with Curvature based step-size","created":"2021-04-15","cited_by_count":0,"area":null},{"id":"2106.08775","title":"Momentum-inspired Low-Rank Coordinate Descent for Diagonally Constrained SDPs","created":"2021-06-16","cited_by_count":0,"area":null},{"id":"2110.08531","title":"A theoretical and empirical study of new adaptive algorithms with additional momentum steps and shifted updates for stochastic non-convex optimization","created":"2021-10-16","cited_by_count":0,"area":null},{"id":"2201.08311","title":"Accelerated Gradient Flow: Risk, Stability, and Implicit Regularization","created":"2022-01-20","cited_by_count":0,"area":null},{"id":"2208.00208","title":"DRSOM: A Dimension Reduced Second-Order Method","created":"2022-07-30","cited_by_count":0,"area":null},{"id":"2208.03941","title":"Provable Acceleration of Nesterov's Accelerated Gradient Method over Heavy Ball Method in Training Over-Parameterized Neural Networks","created":"2022-08-08","cited_by_count":0,"area":null},{"id":"2209.14086","title":"Momentum Gradient Descent Federated Learning with Local Differential Privacy","created":"2022-09-28","cited_by_count":0,"area":null},{"id":"2303.00179","title":"A Unified Momentum-based Paradigm of Decentralized SGD for Non-Convex Models and Heterogeneous Data","created":"2023-03-01","cited_by_count":0,"area":null},{"id":"2308.15434","title":"Random feature approximation for general spectral methods","created":"2023-08-29","cited_by_count":0,"area":null},{"id":"2609.08656","title":"A Lower Bound for the Heavy-Ball Method on Smooth Convex Functions","created":"2026-09-08","cited_by_count":0,"area":null},{"id":"2609.39114","title":"The Row Normalization Puzzle in Muon","created":"2026-09-30","cited_by_count":0,"area":null},{"id":"2610.20413","title":"When Does Momentum Help in Reinforcement Learning with Verifiable Rewards?","created":"2026-10-14","cited_by_count":0,"area":null}],"next_cursor":null}