{"total":156,"rows":[{"id":"2103.00065","title":"Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability","created":"2021-02-26","cited_by_count":427,"area":null},{"id":"2207.14484","title":"Adaptive Gradient Methods at the Edge of Stability","created":"2022-07-29","cited_by_count":101,"area":null},{"id":"2208.11195","title":"Robustness to Unbounded Smoothness of Generalized SignSGD","created":"2022-08-23","cited_by_count":81,"area":null},{"id":"2106.14343","title":"High-probability Bounds for Non-Convex Stochastic Optimization with Heavy Tails","created":"2021-06-28","cited_by_count":80,"area":null},{"id":"2202.05791","title":"The Power of Adaptivity in SGD: Self-Tuning Step Sizes with Unbounded Gradients and Affine Variance","created":"2022-02-11","cited_by_count":61,"area":null},{"id":"2305.18471","title":"Convergence of AdaGrad for Non-convex Objectives: Simple Proofs and Relaxed Assumptions","created":"2023-05-29","cited_by_count":57,"area":null},{"id":"2306.00204","title":"Toward Understanding Why Adam Converges Faster Than SGD for Transformers","created":"2023-05-31","cited_by_count":53,"area":null},{"id":"2112.03459","title":"A Novel Convergence Analysis for Algorithms of the Adam Family","created":"2021-12-07","cited_by_count":50,"area":null},{"id":"2302.14843","title":"High Probability Convergence of Stochastic Gradient Methods","created":"2023-02-28","cited_by_count":49,"area":null},{"id":"2202.00089","title":"Understanding AdamW through Proximal Methods and Scale-Freeness","created":"2022-01-31","cited_by_count":46,"area":null},{"id":"2302.03775","title":"Optimal Stochastic Non-smooth Non-convex Optimization through Online-to-Non-convex Conversion","created":"2023-02-07","cited_by_count":45,"area":null},{"id":"2310.07831","title":"Optimal Linear Decay Learning Rate Schedules and Further Refinements","created":"2023-10-11","cited_by_count":43,"area":null},{"id":"2302.06570","title":"Beyond Uniform Smoothness: A Stopped Analysis of Adaptive SGD","created":"2023-02-13","cited_by_count":41,"area":null},{"id":"2106.08208","title":"SUPER-ADAM: Faster and Universal Framework of Adaptive Gradients","created":"2021-06-15","cited_by_count":40,"area":null},{"id":"2204.02833","title":"High Probability Bounds for a Class of Nonconvex Algorithms with AdaGrad Stepsize","created":"2022-04-06","cited_by_count":40,"area":null},{"id":"2202.04428","title":"Adapting to Mixing Time in Stochastic Optimization with Markovian Data","created":"2022-02-09","cited_by_count":36,"area":null},{"id":"2203.15104","title":"FedADMM: A Federated Primal-Dual Algorithm Allowing Partial Participation","created":"2022-03-28","cited_by_count":36,"area":null},{"id":"2305.16284","title":"DoWG Unleashed: An Efficient Universal Parameter-Free Gradient Descent Method","created":"2023-05-25","cited_by_count":35,"area":null},{"id":"2106.11396","title":"BiAdam: Fast Adaptive Bilevel Optimization Methods","created":"2021-06-21","cited_by_count":33,"area":null},{"id":"2302.08783","title":"SGD with AdaGrad Stepsizes: Full Adaptivity with High Probability to Unknown Parameters, Unbounded Gradients and Affine Variance","created":"2023-02-17","cited_by_count":30,"area":null},{"id":"2101.11075","title":"Adaptivity without Compromise: A Momentumized, Adaptive, Dual Averaged Gradient Method for Stochastic Optimization","created":"2021-01-26","cited_by_count":28,"area":null},{"id":"2102.00236","title":"Parameter-free Stochastic Optimization of Variationally Coherent Functions","created":"2021-01-30","cited_by_count":27,"area":null},{"id":"2402.01567","title":"Understanding Adam Optimizer via Online Learning of Updates: Adam is FTRL in Disguise","created":"2024-02-02","cited_by_count":26,"area":null},{"id":"2101.05471","title":"Towards Practical Adam: Non-Convexity, Convergence Theory, and Mini-Batch Acceleration","created":"2021-01-14","cited_by_count":25,"area":null},{"id":"2102.05320","title":"An Adaptive Stochastic Sequential Quadratic Programming with Differentiable Exact Augmented Lagrangians","created":"2021-02-10","cited_by_count":25,"area":null},{"id":"2109.05109","title":"Toward Communication Efficient Adaptive Gradient Method","created":"2021-09-10","cited_by_count":25,"area":null},{"id":"2206.07082","title":"Stability and Generalization of Stochastic Optimization with Nonconvex and Nonsmooth Problems","created":"2022-06-14","cited_by_count":25,"area":null},{"id":"2211.15120","title":"Improved Representation of Asymmetrical Distances with Interval Quasimetric Embeddings","created":"2022-11-28","cited_by_count":24,"area":null},{"id":"2305.12475","title":"Two Sides of One Coin: the Limits of Untuned SGD and the Power of Adaptive Methods","created":"2023-05-21","cited_by_count":23,"area":null},{"id":"2308.00522","title":"Efficient Federated Learning via Local Adaptive Amended Optimizer with Linear Speedup","created":"2023-07-30","cited_by_count":23,"area":null},{"id":"2509.02981","title":"AdaGrad Meets Muon: Adaptive Stepsizes for Orthogonal Updates","created":"2025-09-03","cited_by_count":23,"area":null},{"id":"2110.15412","title":"Stochastic Mirror Descent: Convergence Analysis and Adaptive Variants via the Mirror Stochastic Polyak Stepsize","created":"2021-10-28","cited_by_count":22,"area":null},{"id":"2406.15244","title":"AdaGrad under Anisotropic Smoothness","created":"2024-06-21","cited_by_count":22,"area":null},{"id":"2402.00949","title":"Geometry of Polynomial Neural Networks","created":"2024-02-01","cited_by_count":21,"area":null},{"id":"2402.03982","title":"On Convergence of Adam for Stochastic Optimization under Relaxed Assumptions","created":"2024-02-06","cited_by_count":21,"area":null},{"id":"2308.05621","title":"Normalized Gradients for All","created":"2023-08-10","cited_by_count":20,"area":null},{"id":"2402.04513","title":"Online Cascade Learning for Efficient Inference over Streams","created":"2024-02-07","cited_by_count":20,"area":null},{"id":"2110.06081","title":"On Expressivity and Trainability of Quadratic Networks","created":"2021-10-12","cited_by_count":19,"area":null},{"id":"2301.04431","title":"Adaptive proximal algorithms for convex optimization under local Lipschitz continuity of the gradient","created":"2023-01-11","cited_by_count":19,"area":null},{"id":"2402.07114","title":"Towards Quantifying the Preconditioning Effect of Adam","created":"2024-02-11","cited_by_count":19,"area":null},{"id":"2310.10082","title":"A simple uniformly optimal method without line search for convex optimization","created":"2023-10-16","cited_by_count":18,"area":null},{"id":"2411.02853","title":"ADOPT: Modified Adam Can Converge with Any $\\beta_2$ with the Optimal Rate","created":"2024-11-05","cited_by_count":18,"area":null},{"id":"2109.03194","title":"On the Convergence of Decentralized Adaptive Gradient Methods","created":"2021-09-07","cited_by_count":17,"area":null},{"id":"2110.01663","title":"Global Convergence and Stability of Stochastic Gradient Descent","created":"2021-10-04","cited_by_count":17,"area":null},{"id":"2206.10189","title":"A General Theory for Federated Optimization with Asynchronous and Heterogeneous Clients Updates","created":"2022-06-21","cited_by_count":17,"area":null},{"id":"2110.11442","title":"Towards Noise-adaptive, Problem-adaptive (Accelerated) Stochastic Gradient Descent","created":"2021-10-21","cited_by_count":16,"area":null},{"id":"2206.00743","title":"Nest Your Adaptive Algorithm for Parameter-Agnostic Nonconvex Minimax Optimization","created":"2022-06-01","cited_by_count":16,"area":null},{"id":"2406.04592","title":"Provable Complexity Improvement of AdaGrad over SGD: Upper and Lower Bounds in Stochastic Non-Convex Optimization","created":"2024-06-07","cited_by_count":16,"area":null},{"id":"2102.09700","title":"AI-SARAH: Adaptive and Implicit Stochastic Recursive Gradient Methods","created":"2021-02-19","cited_by_count":15,"area":null},{"id":"2211.01851","title":"Adaptive Stochastic Variance Reduction for Non-convex Finite-Sum Minimization","created":"2022-11-03","cited_by_count":15,"area":null},{"id":"2402.13794","title":"Revisiting Convergence of AdaGrad with Relaxed Assumptions","created":"2024-02-21","cited_by_count":15,"area":null},{"id":"2209.08005","title":"Stability and Generalization for Markov Chain Stochastic Gradient Methods","created":"2022-09-16","cited_by_count":14,"area":null},{"id":"2104.14840","title":"Unified Convergence Analysis for Adaptive Optimization with Moving Average Estimator","created":"2021-04-30","cited_by_count":13,"area":null},{"id":"2106.16101","title":"AdaGDA: Faster Adaptive Gradient Descent Ascent Methods for Minimax Optimization","created":"2021-06-30","cited_by_count":13,"area":null},{"id":"2111.06328","title":"Stationary Behavior of Constant Stepsize SGD Type Algorithms: An Asymptotic Characterization","created":"2021-11-11","cited_by_count":13,"area":null},{"id":"2209.14827","title":"On the Convergence of AdaGrad(Norm) on $\\R^{d}$: Beyond Convexity, Non-Asymptotic Rate and Acceleration","created":"2022-09-29","cited_by_count":13,"area":null},{"id":"2402.07793","title":"Tuning-Free Stochastic Optimization","created":"2024-02-12","cited_by_count":13,"area":null},{"id":"2406.04142","title":"Stochastic Polyak Step-sizes and Momentum: Convergence Guarantees and Practical Performance","created":"2024-06-06","cited_by_count":13,"area":null},{"id":"2506.04805","title":"Adaptive Preconditioners Trigger Loss Spikes in Adam","created":"2025-06-05","cited_by_count":13,"area":null},{"id":"2206.03907","title":"A Unified Convergence Theorem for Stochastic Optimization Methods","created":"2022-06-08","cited_by_count":12,"area":null},{"id":"2402.03126","title":"How Free is Parameter-Free Stochastic Optimization?","created":"2024-02-05","cited_by_count":12,"area":null},{"id":"2506.23803","title":"SGD with Adaptive Preconditioning: Unified Analysis and Momentum Acceleration","created":"2025-06-30","cited_by_count":12,"area":null},{"id":"2102.07002","title":"On the Last Iterate Convergence of Momentum Methods","created":"2021-02-13","cited_by_count":10,"area":null},{"id":"2108.11483","title":"Heavy-tailed Streaming Statistical Estimation","created":"2021-08-25","cited_by_count":10,"area":null},{"id":"2109.14213","title":"On the One-sided Convergence of Adam-type Algorithms in Non-convex Non-concave Min-max Optimization","created":"2021-09-29","cited_by_count":9,"area":null},{"id":"2206.06900","title":"Grad-GradaGrad? A Non-Monotone Adaptive Stochastic Gradient Method","created":"2022-06-14","cited_by_count":9,"area":null},{"id":"2306.02527","title":"Searching for Optimal Per-Coordinate Step-sizes with Multidimensional Backtracking","created":"2023-06-05","cited_by_count":9,"area":null},{"id":"2310.11291","title":"An Automatic Learning Rate Schedule Algorithm for Achieving Faster Convergence and Steeper Descent","created":"2023-10-17","cited_by_count":9,"area":null},{"id":"2401.01148","title":"PAC-Bayes-Chernoff bounds for unbounded losses","created":"2024-01-02","cited_by_count":9,"area":null},{"id":"2110.09057","title":"Training Deep Neural Networks with Adaptive Momentum Inspired by the Quadratic Optimization","created":"2021-10-18","cited_by_count":8,"area":null},{"id":"2203.01400","title":"Adaptive Gradient Methods with Local Guarantees","created":"2022-03-02","cited_by_count":8,"area":null},{"id":"2210.17478","title":"TiAda: A Time-scale Adaptive Algorithm for Nonconvex Minimax Optimization","created":"2022-10-31","cited_by_count":8,"area":null},{"id":"2311.02000","title":"High Probability Convergence of Adam Under Unbounded Gradients and Affine Variance Noise","created":"2023-11-03","cited_by_count":8,"area":null},{"id":"2402.11215","title":"AdAdaGrad: Adaptive Batch Size Schemes for Adaptive Gradient Methods","created":"2024-02-17","cited_by_count":8,"area":null},{"id":"2403.02648","title":"Remove that Square Root: A New Efficient Scale-Invariant Version of AdaGrad","created":"2024-03-05","cited_by_count":8,"area":null},{"id":"2201.11204","title":"On the Convergence of mSGD and AdaGrad for Stochastic Optimization","created":"2022-01-26","cited_by_count":7,"area":null},{"id":"2203.15797","title":"Convergence of First-Order Methods for Constrained Nonconvex Optimization with Dependent Data","created":"2022-03-29","cited_by_count":7,"area":null},{"id":"2208.05287","title":"Adaptive Learning Rates for Faster Stochastic Gradient Methods","created":"2022-08-10","cited_by_count":7,"area":null},{"id":"2307.06306","title":"Locally Adaptive Federated Learning","created":"2023-07-12","cited_by_count":7,"area":null},{"id":"2310.02093","title":"Stochastic Gradient Descent with Preconditioned Polyak Step-size","created":"2023-10-03","cited_by_count":7,"area":null},{"id":"2410.15637","title":"Large Deviation Upper Bounds and Improved MSE Rates of Nonlinear SGD: Heavy-tailed Noise and Power of Symmetry","created":"2024-10-21","cited_by_count":7,"area":null},{"id":"2511.06086","title":"MuonAll: Muon Variant for Efficient Finetuning of Large Language Models","created":"2025-11-08","cited_by_count":7,"area":null},{"id":"2310.14085","title":"Adaptive, Doubly Optimal No-Regret Learning in Strongly Monotone and Exp-Concave Games with Gradient Feedback","created":"2023-10-21","cited_by_count":6,"area":null},{"id":"2505.20219","title":"New Perspectives on the Polyak Stepsize: Surrogate Functions and Negative Results","created":"2025-05-26","cited_by_count":6,"area":null},{"id":"2110.07435","title":"Adaptive Differentially Private Empirical Risk Minimization","created":"2021-10-14","cited_by_count":5,"area":null},{"id":"2211.07303","title":"Adaptive Federated Minimax Optimization with Lower Complexities","created":"2022-11-14","cited_by_count":5,"area":null},{"id":"2307.13680","title":"High Probability Analysis for Non-Convex Stochastic Optimization with Clipping","created":"2023-07-25","cited_by_count":5,"area":null},{"id":"2312.17369","title":"SANIA: Polyak-type Optimization Framework Leads to Scale Invariant Stochastic Algorithms","created":"2023-12-28","cited_by_count":5,"area":null},{"id":"2507.09823","title":"Nesterov Finds GRAAL: Optimal and Adaptive Gradient Method for Convex Optimization","created":"2025-07-13","cited_by_count":5,"area":null},{"id":"2105.06543","title":"Policy Optimization in Dynamic Bayesian Network Hybrid Models of Biomanufacturing Processes","created":"2021-05-13","cited_by_count":4,"area":null},{"id":"2107.08011","title":"Adaptive first-order methods revisited: Convex optimization without Lipschitz requirements","created":"2021-07-16","cited_by_count":4,"area":null},{"id":"2109.08282","title":"AdaLoss: A computationally-efficient and provably convergent adaptive gradient method","created":"2021-09-17","cited_by_count":4,"area":null},{"id":"2110.05454","title":"Momentum Centering and Asynchronous Update for Adaptive Gradient Methods","created":"2021-10-11","cited_by_count":4,"area":null},{"id":"2111.01040","title":"STORM+: Fully Adaptive SGD with Momentum for Nonconvex Optimization","created":"2021-11-01","cited_by_count":4,"area":null},{"id":"2202.10640","title":"Convergence of online $k$-means","created":"2022-02-22","cited_by_count":4,"area":null},{"id":"2305.18666","title":"BiSLS/SPS: Auto-tune Step Sizes for Stable Bi-level Optimization","created":"2023-05-30","cited_by_count":4,"area":null},{"id":"2402.06271","title":"Adaptive proximal gradient methods are universal without approximation","created":"2024-02-09","cited_by_count":4,"area":null},{"id":"2411.05746","title":"Continuous-Time Analysis of Adaptive Optimization and Normalization","created":"2024-11-08","cited_by_count":4,"area":null},{"id":"2603.09581","title":"Towards Understanding Adam Convergence on Highly Degenerate Polynomials","created":"2026-03-10","cited_by_count":4,"area":null},{"id":"2103.03570","title":"Second-order step-size tuning of SGD for non-convex optimization","created":"2021-03-05","cited_by_count":3,"area":null}],"next_cursor":"100"}