{"list":"other","total":63,"rows":[{"id":"1405.4980","title":"Convex Optimization: Algorithms and Complexity","created":"2014-05-20","cited_by_count":575},{"id":"1802.04434","title":"signSGD: Compressed Optimisation for Non-Convex Problems","created":"2018-02-13","cited_by_count":571},{"id":"1809.07599","title":"Sparsified SGD with Memory","created":"2018-09-20","cited_by_count":376},{"id":"1402.4102","title":"Stochastic Gradient Hamiltonian Monte Carlo","created":"2014-02-17","cited_by_count":343},{"id":"1905.11881","title":"Why gradient clipping accelerates training: A theoretical justification for adaptivity","created":"2019-05-28","cited_by_count":341},{"id":"1704.04289","title":"Stochastic Gradient Descent as Approximate Bayesian Inference","created":"2017-04-13","cited_by_count":300},{"id":"1711.04623","title":"Three Factors Influencing Minima in SGD","created":"2017-11-13","cited_by_count":273},{"id":"1901.06053","title":"A Tail-Index Analysis of Stochastic Gradient Noise in Deep Neural Networks","created":"2019-01-18","cited_by_count":214},{"id":"1503.01243","title":"A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights","created":"2015-03-04","cited_by_count":206},{"id":"2101.12176","title":"On the Origin of Implicit Regularization in Stochastic Gradient Descent","created":"2021-01-28","cited_by_count":184},{"id":"1511.06251","title":"Stochastic modified equations and adaptive stochastic gradient algorithms","created":"2015-11-19","cited_by_count":142},{"id":"1705.07774","title":"Dissecting Adam: The Sign, Magnitude and Variance of Stochastic Gradients","created":"2017-05-22","cited_by_count":139},{"id":"2106.05203","title":"EF21: A New, Simpler, Theoretically Better, and Practically Faster Error Feedback","created":"2021-06-09","cited_by_count":133},{"id":"2010.05627","title":"Towards Theoretically Understanding Why SGD Generalizes Better Than ADAM in Deep Learning","created":"2020-10-12","cited_by_count":126},{"id":"1811.01558","title":"Stochastic Modified Equations and Dynamics of Stochastic Gradient Algorithms I: Mathematical Foundations","created":"2018-11-05","cited_by_count":114},{"id":"1901.09269","title":"Distributed Learning with Compressed Gradient Differences","created":"2019-01-26","cited_by_count":112},{"id":"2205.10287","title":"On the SDEs and Scaling Rules for Adaptive Gradient Algorithms","created":"2022-05-20","cited_by_count":104},{"id":"2304.13972","title":"Convergence of Adam Under Relaxed Assumptions","created":"2023-04-27","cited_by_count":99},{"id":"2010.02519","title":"Improved Analysis of Clipping Algorithms for Non-convex Optimization","created":"2020-10-05","cited_by_count":89},{"id":"2204.01050","title":"Understanding the unstable convergence of gradient descent","created":"2022-04-03","cited_by_count":86},{"id":"1905.11261","title":"A Unified Theory of SGD: Variance Reduction, Sampling, Quantization and Coordinate Descent","created":"2019-05-27","cited_by_count":84},{"id":"2305.01588","title":"Revisiting Gradient Clipping: Stochastic bias and tight convergence guarantees","created":"2023-05-02","cited_by_count":83},{"id":"2006.15081","title":"On the Generalization Benefit of Noise in Stochastic Gradient Descent","created":"2020-06-26","cited_by_count":82},{"id":"2208.11195","title":"Robustness to Unbounded Smoothness of Generalized SignSGD","created":"2022-08-23","cited_by_count":81},{"id":"2402.19449","title":"Heavy-Tailed Class Imbalance and Why Adam Outperforms Gradient Descent on Language Models","created":"2024-02-29","cited_by_count":80},{"id":"1602.02666","title":"A Variational Analysis of Stochastic Gradient Algorithms","created":"2016-02-08","cited_by_count":57},{"id":"2305.18471","title":"Convergence of AdaGrad for Non-convex Objectives: Simple Proofs and Relaxed Assumptions","created":"2023-05-29","cited_by_count":57},{"id":"1308.6370","title":"Fast Convergence of Stochastic Gradient Descent under a Strong Growth Condition","created":"2013-08-29","cited_by_count":54},{"id":"1905.12938","title":"Stochastic Sign Descent Methods: New Algorithms and Better Theory","created":"2019-05-30","cited_by_count":49},{"id":"2306.01264","title":"Convex and Non-convex Optimization Under Generalized Smoothness","created":"2023-06-02","cited_by_count":44},{"id":"1912.00018","title":"On the Heavy-Tailed Theory of Stochastic Gradient Descent for Deep Neural Networks","created":"2019-11-29","cited_by_count":43},{"id":"2302.06570","title":"Beyond Uniform Smoothness: A Stopped Analysis of Adaptive SGD","created":"2023-02-13","cited_by_count":41},{"id":"2303.02854","title":"Generalized-Smooth Nonconvex Optimization is As Efficient As Smooth Nonconvex Optimization","created":"2023-03-06","cited_by_count":35},{"id":"1901.09149","title":"Escaping Saddle Points with Adaptive Gradient Methods","created":"2019-01-26","cited_by_count":34},{"id":"2311.03252","title":"Parameter-Agnostic Optimization under Relaxed Smoothness","created":"2023-11-06","cited_by_count":29},{"id":"2103.17182","title":"Positive-Negative Momentum: Manipulating Stochastic Gradient Noise to Improve Generalization","created":"2021-03-31","cited_by_count":28},{"id":"2409.14989","title":"Methods for Convex $(L_0,L_1)$-Smooth Optimization: Clipping, Acceleration, and Adaptivity","created":"2024-09-23","cited_by_count":25},{"id":"2301.08203","title":"An SDE for Modeling SAM: Theory and Insights","created":"2023-01-19","cited_by_count":22},{"id":"2303.00883","title":"Variance-reduced Clipping for Non-convex Optimization","created":"2023-03-02","cited_by_count":22},{"id":"2501.12666","title":"Explicit Eigenvalue Regularization Improves Sharpness-Aware Minimization","created":"2025-01-22","cited_by_count":21},{"id":"2106.05958","title":"High Probability Complexity Bounds for Non-Smooth Stochastic Optimization with Heavy-Tailed Noise","created":"2021-06-10","cited_by_count":19},{"id":"2410.10800","title":"Optimizing $(L_0, L_1)$-Smooth Functions by Gradient Methods","created":"2024-10-14","cited_by_count":19},{"id":"1810.02565","title":"Continuous-time Models for Stochastic Optimization Algorithms","created":"2018-10-05","cited_by_count":17},{"id":"2404.01436","title":"Convergence Guarantees for RMSProp and Adam in Generalized-smooth Non-convex Optimization with Affine Noise Variance","created":"2024-04-01","cited_by_count":14},{"id":"2502.07923","title":"Sign Operator for Coping with Heavy-Tailed Noise in Non-Convex Optimization: High Probability Bounds Under $(L_0, L_1)$-Smoothness","created":"2025-02-11","cited_by_count":13},{"id":"2406.11733","title":"To Clip or not to Clip: the Dynamics of SGD with Gradient Clipping in High-Dimensions","created":"2024-06-17","cited_by_count":7},{"id":"2411.12135","title":"Exact Risk Curves of signSGD in High-Dimensions: Quantifying Preconditioning and Noise-Compression Effects","created":"2024-11-19","cited_by_count":7},{"id":"1911.05206","title":"Shadowing Properties of Optimization Algorithms","created":"2019-11-12","cited_by_count":6},{"id":"2402.12508","title":"SDEs for Minimax Optimization","created":"2024-02-19","cited_by_count":6},{"id":"2502.03401","title":"Revisiting Stochastic Proximal Point Methods: Generalized Smoothness and Similarity","created":"2025-02-05","cited_by_count":6},{"id":"2412.11773","title":"Toward a Unified Theory of Gradient Descent under Generalized Smoothness","created":"2024-12-16","cited_by_count":5},{"id":"2502.00753","title":"Mirror Descent Under Generalized Smoothness","created":"2025-02-02","cited_by_count":5},{"id":"2502.17009","title":"Unbiased and Sign Compression in Distributed Learning: Comparing Noise Resilience via SDEs","created":"2025-02-24","cited_by_count":5},{"id":"2410.14054","title":"Adaptive Gradient Normalization and Independent Sampling for (Stochastic) Generalized-Smooth Optimization","created":"2024-10-17","cited_by_count":4},{"id":"2410.16871","title":"Error Feedback under $(L_0,L_1)$-Smoothness: Normalization and Momentum","created":"2024-10-22","cited_by_count":4},{"id":"2412.02781","title":"Methods with Local Steps and Random Reshuffling for Generally Smooth Non-Convex Federated Optimization","created":"2024-12-03","cited_by_count":4},{"id":"2412.17050","title":"Linear Convergence Rate in Convex Setup is Possible! Gradient Descent Method Variants under $(L_0,L_1)$-Smoothness","created":"2024-12-22","cited_by_count":4},{"id":"2502.08532","title":"Nonlinearly Preconditioned Gradient Methods under Generalized Smoothness","created":"2025-02-12","cited_by_count":4},{"id":"2505.20817","title":"High-Probability Convergence of Clipped SGD under Heavy-Tailed Noise and $(L_0,L_1)$-Smoothness","created":"2025-05-27","cited_by_count":4},{"id":"2501.18198","title":"Power of Generalized Smoothness in Stochastic Convex Optimization: First- and Zero-Order Algorithms","created":"2025-01-30","cited_by_count":1},{"id":"2502.11125","title":"Convergence Analysis of Stochastic Accelerated Gradient Methods for Generalized Smooth Optimizations","created":"2025-02-16","cited_by_count":1},{"id":"2502.16492","title":"Convergence of Clipped SGD on Convex $(L_0,L_1)$-Smooth Functions","created":"2025-02-23","cited_by_count":1},{"id":"2503.04712","title":"Efficiently Escaping Saddle Points under Generalized Smoothness via Self-Bounding Regularity","created":"2025-03-06","cited_by_count":1}],"next_cursor":null}