{"total":158,"rows":[{"id":"2202.09357","title":"ProxSkip: Yes! Local Gradient Steps Provably Lead to Communication Acceleration! Finally!","created":"2022-02-18","cited_by_count":134,"area":null},{"id":"2208.11195","title":"Robustness to Unbounded Smoothness of Generalized SignSGD","created":"2022-08-23","cited_by_count":81,"area":null},{"id":"2107.11433","title":"A general sample complexity analysis of vanilla policy gradient","created":"2021-07-23","cited_by_count":76,"area":null},{"id":"2505.13416","title":"Gluon: Making Muon & Scion Great Again! (Bridging Theory and Practice of LMO-based Optimizers for LLMs)","created":"2025-05-19","cited_by_count":72,"area":null},{"id":"2206.07638","title":"Asynchronous SGD Beats Minibatch SGD Under Arbitrary Delays","created":"2022-06-15","cited_by_count":63,"area":null},{"id":"2107.00052","title":"Stochastic Gradient Descent-Ascent and Consensus Optimization for Smooth Games: Convergence Analysis under Expected Co-coercivity","created":"2021-06-30","cited_by_count":43,"area":null},{"id":"2305.16296","title":"A Guide Through the Zoo of Biased SGD","created":"2023-05-25","cited_by_count":40,"area":null},{"id":"2202.04295","title":"On Almost Sure Convergence Rates of Stochastic Gradient Methods","created":"2022-02-09","cited_by_count":35,"area":null},{"id":"2305.16284","title":"DoWG Unleashed: An Efficient Universal Parameter-Free Gradient Descent Method","created":"2023-05-25","cited_by_count":35,"area":null},{"id":"2102.06704","title":"Proximal and Federated Random Reshuffling","created":"2021-02-12","cited_by_count":33,"area":null},{"id":"2207.04338","title":"Variance Reduced ProxSkip: Algorithm, Theory and Application to Federated Learning","created":"2022-07-09","cited_by_count":33,"area":null},{"id":"2111.08611","title":"Stochastic Extragradient: General Analysis and Improved Rates","created":"2021-11-16","cited_by_count":31,"area":null},{"id":"2103.01447","title":"ZeroSARAH: Efficient Nonconvex Finite-Sum Optimization with Zero Full Gradient Computation","created":"2021-03-02","cited_by_count":29,"area":null},{"id":"2306.03638","title":"Provable convergence guarantees for black-box variational inference","created":"2023-06-04","cited_by_count":29,"area":null},{"id":"2102.09385","title":"Convergence of stochastic gradient descent schemes for Lojasiewicz-landscapes","created":"2021-02-16","cited_by_count":28,"area":null},{"id":"2302.00543","title":"DoCoFL: Downlink Compression for Cross-Device Federated Learning","created":"2023-02-01","cited_by_count":28,"area":null},{"id":"2309.13591","title":"Robust Distributed Learning: Tight Error Bounds and Breakdown Point under Data Heterogeneity","created":"2023-09-24","cited_by_count":26,"area":null},{"id":"2205.13925","title":"DELTA: Diverse Client Sampling for Fasting Federated Learning","created":"2022-05-27","cited_by_count":24,"area":null},{"id":"2305.12475","title":"Two Sides of One Coin: the Limits of Untuned SGD and the Power of Adaptive Methods","created":"2023-05-21","cited_by_count":23,"area":null},{"id":"2212.14370","title":"Can 5th Generation Local Training Methods Support Client Sampling? Yes!","created":"2022-12-29","cited_by_count":22,"area":null},{"id":"2305.15349","title":"On the Convergence of Black-Box Variational Inference","created":"2023-05-24","cited_by_count":22,"area":null},{"id":"2306.02601","title":"Aiming towards the minimizers: fast convergence of SGD for overparametrized problems","created":"2023-06-05","cited_by_count":22,"area":null},{"id":"2402.00949","title":"Geometry of Polynomial Neural Networks","created":"2024-02-01","cited_by_count":21,"area":null},{"id":"2205.12856","title":"Stochastic Second-Order Methods Improve Best-Known Sample Complexity of SGD for Gradient-Dominated Function","created":"2022-05-25","cited_by_count":20,"area":null},{"id":"2106.11851","title":"Stochastic Polyak Stepsize with a Moving Target","created":"2021-06-22","cited_by_count":19,"area":null},{"id":"2306.14969","title":"On the Sample Complexity of Quantum Boltzmann Machine Learning","created":"2023-06-26","cited_by_count":19,"area":null},{"id":"2205.13746","title":"Regularized Gradient Descent Ascent for Two-Player Zero-Sum Markov Games","created":"2022-05-27","cited_by_count":18,"area":null},{"id":"2411.02853","title":"ADOPT: Modified Adam Can Converge with Any $\\beta_2$ with the Optimal Rate","created":"2024-11-05","cited_by_count":18,"area":null},{"id":"2110.01663","title":"Global Convergence and Stability of Stochastic Gradient Descent","created":"2021-10-04","cited_by_count":17,"area":null},{"id":"2504.09951","title":"Towards Weaker Variance Assumptions for Stochastic Optimization","created":"2025-04-14","cited_by_count":17,"area":null},{"id":"2110.11442","title":"Towards Noise-adaptive, Problem-adaptive (Accelerated) Stochastic Gradient Descent","created":"2021-10-21","cited_by_count":16,"area":null},{"id":"2312.02828","title":"Convergence Rates for Stochastic Approximation: Biased Noise with Unbounded Variance, and Applications","created":"2023-12-05","cited_by_count":15,"area":null},{"id":"2402.13794","title":"Revisiting Convergence of AdaGrad with Relaxed Assumptions","created":"2024-02-21","cited_by_count":15,"area":null},{"id":"2112.13097","title":"Faster Rates for Compressed Federated Learning with Client-Variance Reduction","created":"2021-12-24","cited_by_count":14,"area":null},{"id":"2206.05891","title":"Anchor Sampling for Federated Learning with Partial Client Participation","created":"2022-06-13","cited_by_count":14,"area":null},{"id":"2307.11782","title":"Convergence of Adam for Non-convex Objectives: Relaxed Hyperparameters and Non-ergodic Case","created":"2023-07-20","cited_by_count":14,"area":null},{"id":"2404.01436","title":"Convergence Guarantees for RMSProp and Adam in Generalized-smooth Non-convex Optimization with Affine Noise Variance","created":"2024-04-01","cited_by_count":14,"area":null},{"id":"2402.07793","title":"Tuning-Free Stochastic Optimization","created":"2024-02-12","cited_by_count":13,"area":null},{"id":"2404.09816","title":"FedP3: Federated Personalized and Privacy-friendly Network Pruning under Model Heterogeneity","created":"2024-04-15","cited_by_count":13,"area":null},{"id":"2410.08305","title":"Randomized Asymmetric Chain of LoRA: The First Meaningful Theoretical Framework for Low-Rank Adaptation","created":"2024-10-10","cited_by_count":13,"area":null},{"id":"2111.11556","title":"FLIX: A Simple and Communication-Efficient Alternative to Local Methods in Federated Learning","created":"2021-11-22","cited_by_count":12,"area":null},{"id":"2206.03907","title":"A Unified Convergence Theorem for Stochastic Optimization Methods","created":"2022-06-08","cited_by_count":12,"area":null},{"id":"2410.12935","title":"Quantum Boltzmann machine learning of ground-state energies","created":"2024-10-16","cited_by_count":12,"area":null},{"id":"2305.17581","title":"Knowledge Distillation Performs Partial Variance Reduction","created":"2023-05-27","cited_by_count":11,"area":"distill"},{"id":"2307.01497","title":"Accelerated stochastic approximation with state-dependent noise","created":"2023-07-04","cited_by_count":11,"area":null},{"id":"2409.08770","title":"Increasing Both Batch Size and Learning Rate Accelerates Stochastic Gradient Descent","created":"2024-09-13","cited_by_count":11,"area":null},{"id":"2506.11214","title":"Complexity of normalized stochastic first-order methods with momentum under heavy-tailed noise","created":"2025-06-12","cited_by_count":11,"area":null},{"id":"2510.19975","title":"Revisiting Zeroth-Order Optimization: Minimum-Variance Two-Point Estimators and Directionally Aligned Perturbations","created":"2025-10-22","cited_by_count":11,"area":null},{"id":"2503.02225","title":"Sharpness-Aware Minimization: General Analysis and Improved Rates","created":"2025-03-04","cited_by_count":10,"area":null},{"id":"2410.04458","title":"A Comprehensive Framework for Analyzing the Convergence of Adam: Bridging the Gap with SGD","created":"2024-10-06","cited_by_count":9,"area":null},{"id":"2110.03294","title":"EF21 with Bells & Whistles: Six Algorithmic Extensions of Modern Error Feedback","created":"2021-10-07","cited_by_count":8,"area":null},{"id":"2206.05124","title":"Stochastic Zeroth order Descent with Structured Directions","created":"2022-06-10","cited_by_count":8,"area":null},{"id":"2302.03550","title":"Exponential convergence rates for momentum stochastic gradient descent in the overparametrized setting","created":"2023-02-07","cited_by_count":8,"area":null},{"id":"2306.03240","title":"Improving Accelerated Federated Learning with Compression and Importance Sampling","created":"2023-06-05","cited_by_count":8,"area":null},{"id":"2307.14528","title":"Function Value Learning: Adaptive Learning Rates Based on the Polyak Stepsize and Function Splitting in ERM","created":"2023-07-26","cited_by_count":8,"area":null},{"id":"2402.11215","title":"AdAdaGrad: Adaptive Batch Size Schemes for Adaptive Gradient Methods","created":"2024-02-17","cited_by_count":8,"area":null},{"id":"2504.06087","title":"Accurate Ab-initio Neural-network Solutions to Large-Scale Electronic Structure Problems","created":"2025-04-08","cited_by_count":8,"area":null},{"id":"2110.11886","title":"Conditionally Gaussian PAC-Bayes","created":"2021-10-22","cited_by_count":7,"area":null},{"id":"2111.13322","title":"Random-reshuffled SARAH does not need a full gradient computations","created":"2021-11-26","cited_by_count":7,"area":null},{"id":"2405.15722","title":"Models That Prove Their Own Correctness","created":"2024-05-24","cited_by_count":7,"area":null},{"id":"2406.12774","title":"Towards Exact Gradient-based Training on Analog In-memory Computing","created":"2024-06-18","cited_by_count":7,"area":null},{"id":"2410.15637","title":"Large Deviation Upper Bounds and Improved MSE Rates of Nonlinear SGD: Heavy-tailed Noise and Power of Symmetry","created":"2024-10-21","cited_by_count":7,"area":null},{"id":"2303.10472","title":"Practical and Matching Gradient Variance Bounds for Black-Box Variational Bayesian Inference","created":"2023-03-18","cited_by_count":6,"area":null},{"id":"2305.17558","title":"Provably Fast Finite Particle Variants of SVGD via Virtual Particle Stochastic Approximation","created":"2023-05-27","cited_by_count":6,"area":null},{"id":"2306.16484","title":"Towards a Better Theoretical Understanding of Independent Subnetwork Training","created":"2023-06-28","cited_by_count":6,"area":null},{"id":"2402.05453","title":"Mitigating Privacy Risk in Membership Inference by Convex-Concave Loss","created":"2024-02-08","cited_by_count":6,"area":"safety"},{"id":"2403.08955","title":"Towards Efficient Risk-Sensitive Policy Gradient: An Iteration Complexity Analysis","created":"2024-03-13","cited_by_count":6,"area":"deeprl"},{"id":"2410.04285","title":"MindFlayer SGD: Efficient Parallel SGD in the Presence of Heterogeneous and Random Worker Compute Times","created":"2024-10-05","cited_by_count":6,"area":null},{"id":"2210.16400","title":"Flatter, faster: scaling momentum for optimal speedup of SGD","created":"2022-10-28","cited_by_count":5,"area":null},{"id":"2311.11841","title":"High Probability Guarantees for Random Reshuffling","created":"2023-11-20","cited_by_count":5,"area":null},{"id":"2402.02857","title":"Non-asymptotic Analysis of Biased Adaptive Stochastic Approximation","created":"2024-02-05","cited_by_count":5,"area":null},{"id":"2402.15344","title":"Iteration and Stochastic First-order Oracle Complexities of Stochastic Gradient Descent using Constant and Decaying Learning Rates","created":"2024-02-23","cited_by_count":5,"area":null},{"id":"2405.13592","title":"Almost sure convergence rates of stochastic gradient methods under gradient domination","created":"2024-05-22","cited_by_count":5,"area":"deeprl"},{"id":"2405.15545","title":"Freya PAGE: First Optimal Time Complexity for Large-Scale Nonconvex Finite-Sum Optimization with Heterogeneous Asynchronous Computations","created":"2024-05-24","cited_by_count":5,"area":null},{"id":"2602.05531","title":"Solving Stochastic Variational Inequalities without the Bounded Variance Assumption","created":"2026-02-05","cited_by_count":5,"area":null},{"id":"2604.16620","title":"Lower Bounds and Proximally Anchored SGD for Non-Convex Minimization Under Unbounded Variance","created":"2026-04-17","cited_by_count":5,"area":null},{"id":"2301.12677","title":"Distributed Stochastic Optimization under a General Variance Condition","created":"2023-01-30","cited_by_count":4,"area":null},{"id":"2303.04622","title":"ELF: Federated Langevin Algorithms with Primal, Dual and Bidirectional Compression","created":"2023-03-08","cited_by_count":4,"area":null},{"id":"2401.10989","title":"Provably Scalable Black-Box Variational Inference with Structured Variational Families","created":"2024-01-19","cited_by_count":4,"area":null},{"id":"2405.15941","title":"A Unified Theory of Stochastic Proximal Point Methods without Smoothness","created":"2024-05-24","cited_by_count":4,"area":null},{"id":"2405.19463","title":"Stochastic Optimization Algorithms for Instrumental Variable Regression with Streaming Data","created":"2024-05-29","cited_by_count":4,"area":null},{"id":"2406.13936","title":"Communication-Efficient Adaptive Batch Size Strategies for Distributed Local Gradient Methods","created":"2024-06-20","cited_by_count":4,"area":null},{"id":"2412.21124","title":"Adaptive Batch Size Schedules for Distributed Training of Language Models with Data and Model Parallelism","created":"2024-12-30","cited_by_count":4,"area":null},{"id":"2509.07972","title":"Theoretical Analysis on how Learning Rate Warmup Accelerates Convergence","created":"2025-09-09","cited_by_count":4,"area":null},{"id":"2208.11879","title":"A simplified convergence theory for Byzantine resilient stochastic gradient descent","created":"2022-08-25","cited_by_count":3,"area":null},{"id":"2308.09310","title":"Variance reduction techniques for stochastic proximal point algorithms","created":"2023-08-18","cited_by_count":3,"area":null},{"id":"2407.01825","title":"Reevaluating Theoretical Analysis Methods for Optimization in Deep Learning","created":"2024-07-01","cited_by_count":3,"area":null},{"id":"2505.11434","title":"Controlling the Flow: Stability and Convergence for Stochastic Gradient Descent with Decaying Regularization","created":"2025-05-16","cited_by_count":3,"area":null},{"id":"2510.10690","title":"Second-order Optimization under Heavy-Tailed Noise: Hessian Clipping and Sample Complexity Limits","created":"2025-10-12","cited_by_count":3,"area":null},{"id":"2205.12549","title":"Learning from time-dependent streaming data with online stochastic algorithms","created":"2022-05-25","cited_by_count":2,"area":null},{"id":"2206.05869","title":"On the Convergence to a Global Solution of Shuffling-Type Gradient Algorithms","created":"2022-06-13","cited_by_count":2,"area":null},{"id":"2303.11602","title":"Convergence of variational Monte Carlo simulation and scale-invariant pre-training","created":"2023-03-21","cited_by_count":2,"area":null},{"id":"2303.16241","title":"Convergence of the Stochastic Heavy Ball Method With Approximate Gradients and/or Block Updating","created":"2023-03-28","cited_by_count":2,"area":null},{"id":"2305.05675","title":"UAdam: Unified Adam-Type Algorithmic Framework for Non-Convex Stochastic Optimization","created":"2023-05-09","cited_by_count":2,"area":null},{"id":"2306.03626","title":"Understanding Progressive Training Through the Framework of Randomized Coordinate Descent","created":"2023-06-06","cited_by_count":2,"area":null},{"id":"2310.12969","title":"Demystifying the Myths and Legends of Nonconvex Convergence of SGD","created":"2023-10-19","cited_by_count":2,"area":null},{"id":"2401.09665","title":"Accelerating Distributed Stochastic Optimization via Self-Repellent Random Walks","created":"2024-01-18","cited_by_count":2,"area":null},{"id":"2403.00853","title":"Parallel Momentum Methods Under Biased Gradient Estimations","created":"2024-02-29","cited_by_count":2,"area":null},{"id":"2403.02347","title":"On the Convergence of Federated Learning Algorithms without Data Similarity","created":"2024-02-29","cited_by_count":2,"area":null},{"id":"2403.06677","title":"Streamlining in the Riemannian Realm: Efficient Riemannian Optimization with Loopless Variance Reduction","created":"2024-03-11","cited_by_count":2,"area":null}],"next_cursor":"100"}