{"total":48,"rows":[{"id":"2202.11946","title":"Temporal Efficient Training of Spiking Neural Network via Gradient Re-weighting","created":"2022-02-24","cited_by_count":198,"area":null},{"id":"2409.03137","title":"The AdEMAMix Optimizer: Better, Faster, Older","created":"2024-09-05","cited_by_count":54,"area":null},{"id":"2507.07101","title":"Small Batch Size Training for Language Models: When Vanilla SGD Works, and Why Gradient Accumulation Is Wasteful","created":"2025-07-09","cited_by_count":41,"area":null},{"id":"2207.05931","title":"Towards understanding how momentum improves generalization in deep learning","created":"2022-07-13","cited_by_count":40,"area":null},{"id":"2405.16002","title":"Does SGD really happen in tiny subspaces?","created":"2024-05-25","cited_by_count":36,"area":null},{"id":"2206.01029","title":"Trajectory of Mini-Batch Momentum: Batch Size Saturation and Convergence in High Dimensions","created":"2022-06-02","cited_by_count":19,"area":null},{"id":"2306.09000","title":"When and Why Momentum Accelerates SGD:An Empirical Study","created":"2023-06-15","cited_by_count":19,"area":null},{"id":"2307.15196","title":"The Marginal Value of Momentum for Small Learning Rate SGD","created":"2023-07-27","cited_by_count":18,"area":null},{"id":"2106.03696","title":"Dynamics of Stochastic Momentum Methods on Large-scale, Quadratic Models","created":"2021-06-07","cited_by_count":16,"area":null},{"id":"2312.12670","title":"On the Role of Server Momentum in Federated Learning","created":"2023-12-19","cited_by_count":16,"area":null},{"id":"2305.15938","title":"First Order Methods with Markovian Noise: from Acceleration to Variational Inequalities","created":"2023-05-25","cited_by_count":14,"area":null},{"id":"2607.21716","title":"A Defense of the Quadratic Model","created":"2026-07-23","cited_by_count":14,"area":null},{"id":"2206.07553","title":"On the fast convergence of minibatch heavy ball momentum","created":"2022-06-15","cited_by_count":13,"area":null},{"id":"2406.04142","title":"Stochastic Polyak Step-sizes and Momentum: Convergence Guarantees and Practical Performance","created":"2024-06-06","cited_by_count":13,"area":null},{"id":"2206.11124","title":"A view of mini-batch SGD via generating functions: conditions of convergence, phase transitions, benefit from negative momenta","created":"2022-06-22","cited_by_count":11,"area":null},{"id":"2102.07002","title":"On the Last Iterate Convergence of Momentum Methods","created":"2021-02-13","cited_by_count":10,"area":null},{"id":"2604.14108","title":"Momentum Further Constrains Sharpness at the Edge of Stochastic Stability","created":"2026-04-15","cited_by_count":10,"area":null},{"id":"2305.17665","title":"Acceleration of stochastic gradient descent with momentum by averaging: finite-sample rates and asymptotic normality","created":"2023-05-28","cited_by_count":9,"area":null},{"id":"2312.14567","title":"Accelerated Convergence of Stochastic Heavy Ball Method under Anisotropic Gradient Noise","created":"2023-12-22","cited_by_count":9,"area":null},{"id":"2403.05293","title":"Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks","created":"2024-03-08","cited_by_count":9,"area":null},{"id":"2106.02985","title":"Escaping Saddle Points Faster with Stochastic Momentum","created":"2021-06-05","cited_by_count":8,"area":null},{"id":"2310.15165","title":"Handling Data Heterogeneity via Architectural Design for Federated Visual Recognition","created":"2023-10-23","cited_by_count":8,"area":null},{"id":"2411.19671","title":"On the Performance Analysis of Momentum Method: A Frequency Domain Perspective","created":"2024-11-29","cited_by_count":8,"area":null},{"id":"2111.06171","title":"Convergence and Stability of the Stochastic Proximal Point Algorithm with Momentum","created":"2021-11-11","cited_by_count":7,"area":null},{"id":"2102.11755","title":"Analytical Study of Momentum-Based Acceleration Methods in Paradigmatic High-Dimensional Non-Convex Problems","created":"2021-02-23","cited_by_count":5,"area":null},{"id":"2205.08385","title":"Feedback Gradient Descent: Efficient and Stable Optimization with Orthogonality for DNNs","created":"2022-05-12","cited_by_count":5,"area":null},{"id":"2210.16400","title":"Flatter, faster: scaling momentum for optimal speedup of SGD","created":"2022-10-28","cited_by_count":5,"area":null},{"id":"2403.12511","title":"Projected Forward Gradient-Guided Frank-Wolfe Algorithm via Variance Reduction","created":"2024-03-19","cited_by_count":5,"area":null},{"id":"2604.15416","title":"StoSignSGD: Unbiased Structural Stochasticity Fixes SignSGD for Training Large Language Models","created":"2026-04-16","cited_by_count":5,"area":null},{"id":"2311.11166","title":"From Optimization to Control: Quasi Policy Iteration","created":"2023-11-18","cited_by_count":4,"area":null},{"id":"2311.15051","title":"Gradient Descent with Polyak's Momentum Finds Flatter Minima via Large Catapults","created":"2023-11-25","cited_by_count":3,"area":null},{"id":"2401.09415","title":"Randomized Kaczmarz with geometrically smoothed momentum","created":"2024-01-17","cited_by_count":3,"area":null},{"id":"2603.09355","title":"SHANG++: Robust Stochastic Acceleration under Multiplicative Noise","created":"2026-03-10","cited_by_count":3,"area":null},{"id":"2202.07808","title":"Policy Learning and Evaluation with Randomized Quasi-Monte Carlo","created":"2022-02-16","cited_by_count":2,"area":null},{"id":"2406.13041","title":"Accelerated Stochastic Min-Max Optimization Based on Bias-corrected Momentum","created":"2024-06-18","cited_by_count":2,"area":null},{"id":"2411.07120","title":"Lean and Mean Adaptive Optimization via Subset-Norm and Subspace-Momentum with Convergence Guarantees","created":"2024-11-11","cited_by_count":2,"area":null},{"id":"2501.08883","title":"Increasing Batch Size Improves Convergence of Stochastic Gradient Descent with Momentum","created":"2025-01-15","cited_by_count":2,"area":null},{"id":"2512.02342","title":"Safeguarded Stochastic Polyak Step Sizes for Non-smooth Optimization: Robust Performance Without Small (Sub)Gradients","created":"2025-12-02","cited_by_count":2,"area":null},{"id":"2606.19179","title":"Compute Efficiency and Serial Runtime Tradeoffs for Stochastic Momentum Methods","created":"2026-06-17","cited_by_count":2,"area":null},{"id":"2106.06753","title":"Scaling transition from momentum stochastic gradient descent to plain stochastic gradient descent","created":"2021-06-12","cited_by_count":1,"area":null},{"id":"2401.06738","title":"(Accelerated) Noise-adaptive Stochastic Heavy-Ball Momentum","created":"2024-01-12","cited_by_count":1,"area":null},{"id":"2508.03105","title":"Accelerating SGDM via Learning Rate and Batch Size Schedules: A Lyapunov-Based Analysis","created":"2025-08-05","cited_by_count":1,"area":null},{"id":"2105.13464","title":"Training With Data Dependent Dynamic Learning Rates","created":"2021-05-27","cited_by_count":0,"area":null},{"id":"2110.15547","title":"Does Momentum Help? A Sample Complexity Analysis","created":"2021-10-29","cited_by_count":0,"area":null},{"id":"2305.18864","title":"Stochastic Gradient Langevin Dynamics Based on Quantization with Increasing Resolution","created":"2023-05-30","cited_by_count":0,"area":null},{"id":"2605.18609","title":"Perfect Parallelization in Mini-Batch SGD with Classical Momentum Acceleration","created":"2026-05-18","cited_by_count":0,"area":null},{"id":"2605.28961","title":"Dynamics of Stochastic Momentum with Sparse Updates in High Dimensions","created":"2026-05-27","cited_by_count":0,"area":null},{"id":"2610.16034","title":"A Nonlinear Time-Series Theory for Constant-step SGD with Momentum","created":"2026-10-12","cited_by_count":0,"area":null}],"next_cursor":null}