{"total":104,"rows":[{"id":"2304.13960","title":"Noise Is Not the Main Factor Behind the Gap Between SGD and Adam on Transformers, but Sign Descent Might Be","created":"2023-04-27","cited_by_count":114,"area":null},{"id":"2305.01588","title":"Revisiting Gradient Clipping: Stochastic bias and tight convergence guarantees","created":"2023-05-02","cited_by_count":83,"area":null},{"id":"2208.11195","title":"Robustness to Unbounded Smoothness of Generalized SignSGD","created":"2022-08-23","cited_by_count":81,"area":null},{"id":"2106.14343","title":"High-probability Bounds for Non-Convex Stochastic Optimization with Heavy Tails","created":"2021-06-28","cited_by_count":80,"area":null},{"id":"2109.14119","title":"Stochastic Training is Not Necessary for Generalization","created":"2021-09-29","cited_by_count":63,"area":null},{"id":"2302.14843","title":"High Probability Convergence of Stochastic Gradient Methods","created":"2023-02-28","cited_by_count":49,"area":null},{"id":"2410.13849","title":"From Gradient Clipping to Normalization for Heavy Tailed SGD","created":"2024-10-17","cited_by_count":49,"area":null},{"id":"2506.04192","title":"Lions and Muons: Optimization via Stochastic Frank-Wolfe under Heavy-Tailed Noise","created":"2025-06-04","cited_by_count":41,"area":null},{"id":"2102.06489","title":"Stability and Convergence of Stochastic Gradient Clipping: Beyond Lipschitz Continuity and Smoothness","created":"2021-02-12","cited_by_count":37,"area":null},{"id":"2102.10346","title":"Convergence Rates of Stochastic Gradient Descent under Infinite Noise Variance","created":"2021-02-20","cited_by_count":35,"area":null},{"id":"2210.14355","title":"Parameter-free Regret in High Probability with Heavy Tails","created":"2022-10-25","cited_by_count":28,"area":null},{"id":"2302.06763","title":"Breaking the Lower Bound with (Little) Structure: Acceleration in Non-Convex Stochastic Optimization with Heavy-Tailed Noise","created":"2023-02-14","cited_by_count":27,"area":null},{"id":"2202.11632","title":"Mirror Descent Strikes Again: Optimal Stochastic Convex Optimization under Infinite Noise Variance","created":"2022-02-23","cited_by_count":24,"area":null},{"id":"2305.03938","title":"Adam-family Methods for Nonsmooth Optimization with Convergence Guarantees","created":"2023-05-06","cited_by_count":22,"area":null},{"id":"2406.15244","title":"AdaGrad under Anisotropic Smoothness","created":"2024-06-21","cited_by_count":22,"area":null},{"id":"2205.05040","title":"A Communication-Efficient Distributed Gradient Clipping Algorithm for Training Deep Neural Networks","created":"2022-05-10","cited_by_count":20,"area":null},{"id":"2303.12277","title":"Stochastic Nonsmooth Convex Optimization with Heavy-Tailed Noises: High-Probability Bound, In-Expectation Rate and Initial Distance Adaptation","created":"2023-03-22","cited_by_count":20,"area":null},{"id":"2410.16561","title":"Revisiting Gradient Normalization and Clipping for Nonconvex SGD under Heavy-Tailed Noise: Necessity, Sufficiency, and Acceleration","created":"2024-10-21","cited_by_count":18,"area":null},{"id":"2202.06915","title":"Stochastic linear optimization never overfits with quadratically-bounded losses on general data","created":"2022-02-14","cited_by_count":17,"area":null},{"id":"2204.02593","title":"Nonlinear gradient mappings and stochastic optimization: A general framework with applications to heavy-tail noise","created":"2022-04-06","cited_by_count":16,"area":null},{"id":"2206.01095","title":"Clipped Stochastic Methods for Variational Inequalities with Heavy-Tailed Noise","created":"2022-06-02","cited_by_count":14,"area":null},{"id":"2502.07923","title":"Sign Operator for Coping with Heavy-Tailed Noise in Non-Convex Optimization: High Probability Bounds Under $(L_0, L_1)$-Smoothness","created":"2025-02-11","cited_by_count":13,"area":null},{"id":"2210.13111","title":"Federated Learning and Meta Learning: Approaches, Applications, and Directions","created":"2022-10-24","cited_by_count":12,"area":null},{"id":"2201.06714","title":"AdaTerm: Adaptive T-Distribution Estimated Robust Moments for Noise-Robust Stochastic Gradient Optimization","created":"2022-01-18","cited_by_count":11,"area":null},{"id":"2305.18929","title":"Clip21: Error Feedback for Gradient Clipping","created":"2023-05-30","cited_by_count":11,"area":null},{"id":"2306.11497","title":"Convergence and concentration properties of constant step-size SGD through Markov chains","created":"2023-06-20","cited_by_count":11,"area":null},{"id":"2402.10898","title":"The Price of Adaptivity in Stochastic Convex Optimization","created":"2024-02-16","cited_by_count":11,"area":null},{"id":"2405.14432","title":"Adaptive Gradient Clipping for Robust Federated Learning","created":"2024-05-23","cited_by_count":11,"area":null},{"id":"2108.11483","title":"Heavy-tailed Streaming Statistical Estimation","created":"2021-08-25","cited_by_count":10,"area":null},{"id":"2208.13592","title":"Smooth Monotone Stochastic Variational Inequalities and Saddle Point Problems: A Survey","created":"2022-08-29","cited_by_count":10,"area":null},{"id":"2210.00690","title":"Taming Fat-Tailed (\"Heavier-Tailed'' with Potentially Infinite Variance) Noise in Federated Learning","created":"2022-10-03","cited_by_count":10,"area":null},{"id":"2603.14315","title":"Enhancing LLM Training via Spectral Clipping","created":"2026-03-15","cited_by_count":10,"area":null},{"id":"2208.08567","title":"High Probability Bounds for Stochastic Subgradient Schemes with Heavy Tailed Noise","created":"2022-08-17","cited_by_count":9,"area":null},{"id":"2211.00969","title":"Large deviations rates for stochastic gradient descent with strongly convex functions","created":"2022-11-02","cited_by_count":9,"area":null},{"id":"2302.07155","title":"EPISODE: Episodic Gradient Clipping with Periodic Resampled Corrections for Federated Learning with Heterogeneous Data","created":"2023-02-14","cited_by_count":7,"area":null},{"id":"2310.18784","title":"High-probability Convergence Bounds for Nonlinear Stochastic Gradient Descent Under Heavy-tailed Noise","created":"2023-10-28","cited_by_count":7,"area":null},{"id":"2312.07142","title":"General Tail Bounds for Non-Smooth Stochastic Mirror Descent","created":"2023-12-12","cited_by_count":7,"area":null},{"id":"2502.04164","title":"Efficient Distributed Optimization under Heavy-Tailed Noise","created":"2025-02-06","cited_by_count":7,"area":null},{"id":"2307.13680","title":"High Probability Analysis for Non-Convex Stochastic Optimization with Clipping","created":"2023-07-25","cited_by_count":5,"area":null},{"id":"2311.11841","title":"High Probability Guarantees for Random Reshuffling","created":"2023-11-20","cited_by_count":5,"area":null},{"id":"2404.10995","title":"Clipped SGD Algorithms for Performative Prediction: Tight Bounds for Clipping Bias and Remedies","created":"2024-04-17","cited_by_count":5,"area":null},{"id":"2405.17529","title":"Clip Body and Tail Separately: High Probability Guarantees for DPSGD with Heavy Tails","created":"2024-05-27","cited_by_count":5,"area":null},{"id":"2101.00935","title":"First-Order Methods for Convex Optimization","created":"2021-01-04","cited_by_count":4,"area":null},{"id":"2106.05397","title":"From inexact optimization to learning via gradient concentration","created":"2021-06-09","cited_by_count":4,"area":null},{"id":"2509.14952","title":"Stochastic Bilevel Optimization with Heavy-Tailed Noise","created":"2025-09-18","cited_by_count":4,"area":null},{"id":"2510.11676","title":"Accelerated stochastic first-order method for convex optimization under heavy-tailed noise","created":"2025-10-13","cited_by_count":4,"area":null},{"id":"2512.13123","title":"Stopping Rules for Stochastic Gradient Descent via Anytime-Valid Confidence Sequences","created":"2025-12-15","cited_by_count":4,"area":null},{"id":"2203.14434","title":"Flexible risk design using bi-directional dispersion","created":"2022-03-28","cited_by_count":3,"area":null},{"id":"2309.17316","title":"Robust Stochastic Optimization via Gradient Quantile Clipping","created":"2023-09-29","cited_by_count":3,"area":null},{"id":"2401.13971","title":"Stochastic Weakly Convex Optimization Beyond Lipschitz Continuity","created":"2024-01-25","cited_by_count":3,"area":null},{"id":"2402.05264","title":"AdaBatchGrad: Combining Adaptive Batch Size and Adaptive Step Size","created":"2024-02-07","cited_by_count":3,"area":null},{"id":"2403.02051","title":"Privacy of SGD under Gaussian or Heavy-Tailed Noise: Guarantees without Gradient Clipping","created":"2024-03-04","cited_by_count":3,"area":null},{"id":"2502.13482","title":"Smoothed Normalization for Efficient Distributed Private Optimization","created":"2025-02-19","cited_by_count":3,"area":null},{"id":"2510.10690","title":"Second-order Optimization under Heavy-Tailed Noise: Hessian Clipping and Sample Complexity Limits","created":"2025-10-12","cited_by_count":3,"area":null},{"id":"2607.09097","title":"Solving Stochastic Fixed-Point Equations with High Probability","created":"2026-07-10","cited_by_count":3,"area":null},{"id":"2205.12549","title":"Learning from time-dependent streaming data with online stochastic algorithms","created":"2022-05-25","cited_by_count":2,"area":null},{"id":"2302.02971","title":"U-Clip: On-Average Unbiased Stochastic Gradient Clipping","created":"2023-02-06","cited_by_count":2,"area":null},{"id":"2411.07120","title":"Lean and Mean Adaptive Optimization via Subset-Norm and Subspace-Momentum with Convergence Guarantees","created":"2024-11-11","cited_by_count":2,"area":null},{"id":"2511.04456","title":"Federated Stochastic Minimax Optimization under Heavy-Tailed Noises","created":"2025-11-06","cited_by_count":2,"area":null},{"id":"2601.19379","title":"Optimal Asynchronous Stochastic Nonconvex Optimization under Heavy-Tailed Noise","created":"2026-01-27","cited_by_count":2,"area":null},{"id":"2602.13413","title":"Why is Normalization Preferred? A Worst-Case Complexity Theory for Stochastically Preconditioned SGD under Heavy-Tailed Noise","created":"2026-02-13","cited_by_count":2,"area":null},{"id":"2603.15059","title":"Muon Converges under Heavy-Tailed Noise: Nonconvex H\\\"{o}lder-Smooth Empirical Risk Minimization","created":"2026-03-16","cited_by_count":2,"area":null},{"id":"2605.11838","title":"Gradient Clipping Beyond Vector Norms: A Spectral Approach for Matrix-Valued Parameters","created":"2026-05-12","cited_by_count":2,"area":null},{"id":"2606.15268","title":"When to use what Schatten-$p$ norm in deep learning?","created":"2026-06-13","cited_by_count":2,"area":null},{"id":"2607.27383","title":"The Convergence Behavior of Adam under Heavy-Tailed Noise","created":"2026-07-29","cited_by_count":2,"area":null},{"id":"2203.06474","title":"Optimizer Amalgamation","created":"2022-03-12","cited_by_count":1,"area":null},{"id":"2403.02873","title":"A General Reduction for High-Probability Analysis with General Light-Tailed Distributions","created":"2024-03-05","cited_by_count":1,"area":null},{"id":"2408.09891","title":"Differential Private Stochastic Optimization with Heavy-tailed Data: Towards Optimal Rates","created":"2024-08-19","cited_by_count":1,"area":null},{"id":"2410.13954","title":"Nonlinear Stochastic Gradient Descent and Heavy-tailed Noise: A Unified Framework and High-probability Guarantees","created":"2024-10-17","cited_by_count":1,"area":null},{"id":"2411.06770","title":"Sketched Adaptive Federated Deep Learning: A Sharp Convergence Analysis","created":"2024-11-11","cited_by_count":1,"area":null},{"id":"2411.16462","title":"Lion Cub: Minimizing Communication Overhead in Distributed Lion","created":"2024-11-25","cited_by_count":1,"area":null},{"id":"2509.13628","title":"Accelerated Gradient Methods with Biased Gradient Estimates: Risk Sensitivity, High-Probability Guarantees, and Large Deviation Bounds","created":"2025-09-17","cited_by_count":1,"area":null},{"id":"2510.01578","title":"Gradient Shaping Beyond Clipping: A Functional Perspective on Update Magnitude Control","created":"2025-10-02","cited_by_count":1,"area":null},{"id":"2510.06141","title":"High-Probability Convergence Guarantees of Decentralized SGD","created":"2025-10-07","cited_by_count":1,"area":null},{"id":"2601.11435","title":"Near-Optimal Decentralized Stochastic Nonconvex Optimization with Heavy-Tailed Noise","created":"2026-01-16","cited_by_count":1,"area":null},{"id":"2602.05657","title":"Tight Long-Term Tail Decay of (Clipped) SGD in Non-Convex Optimization","created":"2026-02-05","cited_by_count":1,"area":null},{"id":"2603.19648","title":"Heavy-Tailed and Long-Range Dependent Noise in Stochastic Approximation: A Finite-Time Analysis","created":"2026-03-20","cited_by_count":1,"area":"deeprl"},{"id":"2101.01178","title":"Advances in Electron Microscopy with Deep Learning","created":"2021-01-04","cited_by_count":0,"area":null},{"id":"2105.11135","title":"Robust learning with anytime-guaranteed feedback","created":"2021-05-24","cited_by_count":0,"area":null},{"id":"2205.02273","title":"An Adaptive Incremental Gradient Method With Support for Non-Euclidean Norms","created":"2022-04-28","cited_by_count":0,"area":null},{"id":"2206.13011","title":"Efficient Private SCO for Heavy-Tailed Data via Averaged Clipping","created":"2022-06-27","cited_by_count":0,"area":null},{"id":"2306.09548","title":"Online Heavy-tailed Change-point detection","created":"2023-06-15","cited_by_count":0,"area":null},{"id":"2306.10278","title":"Adaptive Strategies in Non-convex Optimization","created":"2023-06-17","cited_by_count":0,"area":null},{"id":"2402.12828","title":"Tracking the Median of Gradients with a Stochastic Proximal Point Method","created":"2024-02-20","cited_by_count":0,"area":null},{"id":"2403.12688","title":"SEVEN: Pruning Transformer Model by Reserving Sentinels","created":"2024-03-19","cited_by_count":0,"area":null},{"id":"2405.14741","title":"Subsampled Ensemble Can Improve Generalization Tail Exponentially","created":"2024-05-23","cited_by_count":0,"area":null},{"id":"2507.13283","title":"Stochastic Weakly Convex Optimization Under Heavy-Tailed Noises","created":"2025-07-17","cited_by_count":0,"area":null},{"id":"2512.14686","title":"Bias-Variance Trade-off for Clipped Stochastic First-Order Methods: From Bounded Variance to Infinite Mean","created":"2025-12-16","cited_by_count":0,"area":null},{"id":"2601.19730","title":"Stability and Generalization of Stochastic Nonconvex Optimization under Heavy-Tailed Noise: From Gradient Norm to Goldstein Stationarity","created":"2026-01-27","cited_by_count":0,"area":null},{"id":"2602.18002","title":"Asynchronous Heavy-Tailed Optimization","created":"2026-02-20","cited_by_count":0,"area":null},{"id":"2605.00281","title":"High-Probability Convergence in Decentralized Stochastic Optimization with Gradient Tracking","created":"2026-04-30","cited_by_count":0,"area":null},{"id":"2605.07634","title":"Robust stochastic first order methods in heavy-tailed noise via medoid mini-batch gradient sampling","created":"2026-05-08","cited_by_count":0,"area":null},{"id":"2606.18778","title":"Online Distributional Prediction via Latent Cluster Geometry Under Drift and Corruption","created":"2026-06-17","cited_by_count":0,"area":null},{"id":"2606.22433","title":"Escaping the Variance Trap: Jacobian-Free Dynamics for Root-Finding Bilevel Optimization","created":"2026-06-21","cited_by_count":0,"area":"deeprl"},{"id":"2606.22436","title":"Distribution-Aware Robust Bilevel Optimization: Quantile-Guided Huber Updates in Two-Timescale Stochastic Approximation","created":"2026-06-21","cited_by_count":0,"area":null},{"id":"2606.22466","title":"Federated learning with heavy-tailed gradient noise and communication noise: a variance-reduction based algorithm","created":"2026-06-21","cited_by_count":0,"area":null},{"id":"2607.25785","title":"Variance-Reduced Conditional Gradient Methods under Markovian Sampling for Nonconvex Composite Optimization","created":"2026-07-28","cited_by_count":0,"area":null},{"id":"2609.12785","title":"Convergence of Stochastic Gradient Methods under Heavy-Tailed Noise and H\\\"{o}lder Smoothness","created":"2026-09-11","cited_by_count":0,"area":null},{"id":"2609.13040","title":"Quantile-based Loss Filtering for Outlier-Robust Stochastic Gradient Descent","created":"2026-09-11","cited_by_count":0,"area":null},{"id":"2609.30276","title":"Why Clipping Matters in AdaGrad? Toward a High-Probability Theory under Generalized Smoothness","created":"2026-08-12","cited_by_count":0,"area":null}],"next_cursor":"100"}