{"total":116,"rows":[{"id":"2208.09632","title":"Adam Can Converge Without Any Modification On Update Rules","created":"2022-08-20","cited_by_count":114,"area":null},{"id":"2308.08165","title":"Stochastic Controlled Averaging for Federated Learning with Communication Compression","created":"2023-08-16","cited_by_count":101,"area":null},{"id":"2112.03459","title":"A Novel Convergence Analysis for Algorithms of the Adam Family","created":"2021-12-07","cited_by_count":50,"area":null},{"id":"2502.04664","title":"Implicit Bias of Spectral Descent and Muon on Multiclass Separable Data","created":"2025-02-07","cited_by_count":41,"area":null},{"id":"2104.11981","title":"DecentLaM: Decentralized Momentum SGD for Large-batch Deep Training","created":"2021-04-24","cited_by_count":40,"area":null},{"id":"2204.02833","title":"High Probability Bounds for a Class of Nonconvex Algorithms with AdaGrad Stepsize","created":"2022-04-06","cited_by_count":40,"area":null},{"id":"2207.05931","title":"Towards understanding how momentum improves generalization in deep learning","created":"2022-07-13","cited_by_count":40,"area":null},{"id":"2306.16504","title":"Momentum Benefits Non-IID Federated Learning Simply and Provably","created":"2023-06-28","cited_by_count":38,"area":null},{"id":"2310.17998","title":"Closing the Gap Between the Upper Bound and the Lower Bound of Adam's Iteration Complexity","created":"2023-10-27","cited_by_count":38,"area":null},{"id":"2305.15155","title":"Momentum Provably Improves Error Feedback!","created":"2023-05-24","cited_by_count":31,"area":null},{"id":"2208.09900","title":"Provable Adaptivity of Adam under Non-uniform Smoothness","created":"2022-08-21","cited_by_count":30,"area":null},{"id":"2110.03891","title":"Does Momentum Change the Implicit Regularization on Separable Data?","created":"2021-10-08","cited_by_count":22,"area":null},{"id":"2209.15505","title":"Momentum Tracking: Momentum Acceleration for Decentralized Deep Learning on Heterogeneous Data","created":"2022-09-30","cited_by_count":20,"area":null},{"id":"2306.09000","title":"When and Why Momentum Accelerates SGD:An Empirical Study","created":"2023-06-15","cited_by_count":19,"area":null},{"id":"2307.15196","title":"The Marginal Value of Momentum for Small Learning Rate SGD","created":"2023-07-27","cited_by_count":18,"area":null},{"id":"2312.12670","title":"On the Role of Server Momentum in Federated Learning","created":"2023-12-19","cited_by_count":16,"area":null},{"id":"2404.01436","title":"Convergence Guarantees for RMSProp and Adam in Generalized-smooth Non-convex Optimization with Affine Noise Variance","created":"2024-04-01","cited_by_count":14,"area":null},{"id":"2104.14840","title":"Unified Convergence Analysis for Adaptive Optimization with Moving Average Estimator","created":"2021-04-30","cited_by_count":13,"area":null},{"id":"2206.07553","title":"On the fast convergence of minibatch heavy ball momentum","created":"2022-06-15","cited_by_count":13,"area":null},{"id":"2406.04142","title":"Stochastic Polyak Step-sizes and Momentum: Convergence Guarantees and Practical Performance","created":"2024-06-06","cited_by_count":13,"area":null},{"id":"2506.16659","title":"Memory-Efficient LLM Pretraining via Minimalist Optimizer Design","created":"2025-06-20","cited_by_count":13,"area":null},{"id":"2502.01014","title":"Refining Adaptive Zeroth-Order Optimization at Ease","created":"2025-02-03","cited_by_count":12,"area":null},{"id":"2206.11872","title":"Provable Acceleration of Heavy Ball beyond Quadratics for a Class of Polyak-\\L{}ojasiewicz Functions when the Non-Convexity is Averaged-Out","created":"2022-06-22","cited_by_count":11,"area":null},{"id":"2102.07002","title":"On the Last Iterate Convergence of Momentum Methods","created":"2021-02-13","cited_by_count":10,"area":null},{"id":"2106.03034","title":"Minibatch and Momentum Model-based Methods for Stochastic Weakly Convex Optimization","created":"2021-06-06","cited_by_count":10,"area":null},{"id":"2411.07837","title":"FRUGAL: Memory-Efficient Optimization by Reducing State Overhead for Scalable Training","created":"2024-11-12","cited_by_count":10,"area":null},{"id":"2603.15958","title":"Deriving Hyperparameter Scaling Laws via Modern Optimization Theory","created":"2026-03-16","cited_by_count":10,"area":null},{"id":"2604.14108","title":"Momentum Further Constrains Sharpness at the Edge of Stochastic Stability","created":"2026-04-15","cited_by_count":10,"area":null},{"id":"2108.03947","title":"On the Hyperparameters in Stochastic Gradient Descent with Momentum","created":"2021-08-09","cited_by_count":9,"area":null},{"id":"2305.17665","title":"Acceleration of stochastic gradient descent with momentum by averaging: finite-sample rates and asymptotic normality","created":"2023-05-28","cited_by_count":9,"area":null},{"id":"2312.14567","title":"Accelerated Convergence of Stochastic Heavy Ball Method under Anisotropic Gradient Noise","created":"2023-12-22","cited_by_count":9,"area":null},{"id":"2403.02967","title":"Non-convex Stochastic Composite Optimization with Polyak Momentum","created":"2024-03-05","cited_by_count":9,"area":null},{"id":"2403.05293","title":"Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks","created":"2024-03-08","cited_by_count":9,"area":null},{"id":"2410.04458","title":"A Comprehensive Framework for Analyzing the Convergence of Adam: Bridging the Gap with SGD","created":"2024-10-06","cited_by_count":9,"area":null},{"id":"2110.03294","title":"EF21 with Bells & Whistles: Six Algorithmic Extensions of Modern Error Feedback","created":"2021-10-07","cited_by_count":8,"area":null},{"id":"2411.19671","title":"On the Performance Analysis of Momentum Method: A Frequency Domain Perspective","created":"2024-11-29","cited_by_count":8,"area":null},{"id":"2201.11204","title":"On the Convergence of mSGD and AdaGrad for Stochastic Optimization","created":"2022-01-26","cited_by_count":7,"area":null},{"id":"2403.12229","title":"Fusion Transformer with Object Mask Guidance for Image Forgery Analysis","created":"2024-03-18","cited_by_count":7,"area":null},{"id":"2406.12774","title":"Towards Exact Gradient-based Training on Analog In-memory Computing","created":"2024-06-18","cited_by_count":7,"area":null},{"id":"2305.12056","title":"Uniform-in-Time Wasserstein Stability Bounds for (Noisy) Stochastic Gradient Descent","created":"2023-05-20","cited_by_count":6,"area":null},{"id":"2506.00400","title":"Scaling Textual Gradients via Sampling-Based Momentum","created":"2025-05-31","cited_by_count":6,"area":null},{"id":"2508.12327","title":"Convergence Analysis of the Lion Optimizer in Centralized and Distributed Settings","created":"2025-08-17","cited_by_count":6,"area":null},{"id":"2402.09084","title":"Sobolev Training for Operator Learning","created":"2024-02-14","cited_by_count":5,"area":null},{"id":"2403.12511","title":"Projected Forward Gradient-Guided Frank-Wolfe Algorithm via Variance Reduction","created":"2024-03-19","cited_by_count":5,"area":null},{"id":"2502.02002","title":"The Ball-Proximal (=\"Broximal\") Point Method: a New Algorithm, Convergence Theory, and Applications","created":"2025-02-04","cited_by_count":5,"area":null},{"id":"2312.01432","title":"Fast Dual Subgradient Optimization of the Integrated Transportation Distance Between Stochastic Kernels","created":"2023-12-03","cited_by_count":4,"area":null},{"id":"2505.22549","title":"DES-LOC: Desynced Low Communication Adaptive Optimizers for Training Foundation Models","created":"2025-05-28","cited_by_count":4,"area":null},{"id":"2507.05966","title":"Simple Convergence Proof of Adam From a Sign-like Descent Perspective","created":"2025-07-08","cited_by_count":4,"area":null},{"id":"2307.16120","title":"Deep Unrolling Networks with Recurrent Momentum Acceleration for Nonlinear Inverse Problems","created":"2023-07-30","cited_by_count":3,"area":null},{"id":"2311.18578","title":"Communication-Efficient Heterogeneous Federated Learning with Generalized Heavy-Ball Momentum","created":"2023-11-30","cited_by_count":3,"area":null},{"id":"2410.01410","title":"On the Convergence of FedProx with Extrapolation and Inexact Prox","created":"2024-10-02","cited_by_count":3,"area":null},{"id":"2507.09093","title":"Sharp High-Probability Rates for Nonlinear SGD under Heavy-Tailed Noise via Symmetrization","created":"2025-07-12","cited_by_count":3,"area":null},{"id":"2606.00520","title":"In-Expectation Convergence of Stochastic Gradient Methods under Heavy-Tailed Noise","created":"2026-05-30","cited_by_count":3,"area":null},{"id":"2106.02888","title":"Bandwidth-based Step-Sizes for Non-Convex Stochastic Optimization","created":"2021-06-05","cited_by_count":2,"area":null},{"id":"2209.03705","title":"Losing momentum in continuous-time stochastic optimisation","created":"2022-09-08","cited_by_count":2,"area":null},{"id":"2212.04136","title":"Design and Planning of Flexible Mobile Micro-Grids Using Deep Reinforcement Learning","created":"2022-12-08","cited_by_count":2,"area":null},{"id":"2312.12606","title":"Optimizing Neural Networks with Gradient Lexicase Selection","created":"2023-12-19","cited_by_count":2,"area":null},{"id":"2402.02790","title":"Stable and Robust Deep Learning By Hyperbolic Tangent Exponential Linear Unit (TeLU)","created":"2024-02-05","cited_by_count":2,"area":null},{"id":"2403.00853","title":"Parallel Momentum Methods Under Biased Gradient Estimations","created":"2024-02-29","cited_by_count":2,"area":null},{"id":"2410.02806","title":"Investigating the Impact of Randomness on Reproducibility in Computer Vision: A Study on Applications in Civil Engineering and Medicine","created":"2024-09-19","cited_by_count":2,"area":null},{"id":"2412.18790","title":"Torque-Aware Momentum","created":"2024-12-25","cited_by_count":2,"area":null},{"id":"2501.08883","title":"Increasing Batch Size Improves Convergence of Stochastic Gradient Descent with Momentum","created":"2025-01-15","cited_by_count":2,"area":null},{"id":"2504.04520","title":"Hessian of Perplexity for Large Language Models by PyTorch autograd (Open Source)","created":"2025-04-06","cited_by_count":2,"area":null},{"id":"2506.14929","title":"FedOne: Query-Efficient Federated Learning for Black-box Discrete Prompt Learning","created":"2025-06-17","cited_by_count":2,"area":null},{"id":"2510.05361","title":"MT-DAO: Multi-Timescale Distributed Adaptive Optimizers with Local Updates","created":"2025-10-06","cited_by_count":2,"area":null},{"id":"2511.14501","title":"Improved Convergence in Parameter-Agnostic Error Feedback through Momentum","created":"2025-11-18","cited_by_count":2,"area":null},{"id":"2512.02342","title":"Safeguarded Stochastic Polyak Step Sizes for Non-smooth Optimization: Robust Performance Without Small (Sub)Gradients","created":"2025-12-02","cited_by_count":2,"area":null},{"id":"2602.13413","title":"Why is Normalization Preferred? A Worst-Case Complexity Theory for Stochastically Preconditioned SGD under Heavy-Tailed Noise","created":"2026-02-13","cited_by_count":2,"area":null},{"id":"2605.11838","title":"Gradient Clipping Beyond Vector Norms: A Spectral Approach for Matrix-Valued Parameters","created":"2026-05-12","cited_by_count":2,"area":null},{"id":"2206.00632","title":"Computing the Variance of Shuffling Stochastic Gradient Algorithms via Power Spectral Density Analysis","created":"2022-06-01","cited_by_count":1,"area":null},{"id":"2312.15853","title":"Curricular and Cyclical Loss for Time Series Learning Strategy","created":"2023-12-26","cited_by_count":1,"area":null},{"id":"2312.17517","title":"Embedded feature selection in LSTM networks with multi-objective evolutionary ensemble learning for time series forecasting","created":"2023-12-29","cited_by_count":1,"area":null},{"id":"2401.06738","title":"(Accelerated) Noise-adaptive Stochastic Heavy-Ball Momentum","created":"2024-01-12","cited_by_count":1,"area":null},{"id":"2411.01548","title":"Analysis of regularized federated learning","created":"2024-11-03","cited_by_count":1,"area":null},{"id":"2501.19082","title":"A Bias-Correction Decentralized Stochastic Gradient Algorithm with Momentum Acceleration","created":"2025-01-31","cited_by_count":1,"area":null},{"id":"2506.23544","title":"Both Asymptotic and Non-Asymptotic Convergence of Quasi-Hyperbolic Momentum using Increasing Batch Size","created":"2025-06-30","cited_by_count":1,"area":null},{"id":"2507.19627","title":"Federated Calculation of the Free-Support Transportation Barycenter by Single-Loop Dual Decomposition","created":"2025-07-25","cited_by_count":1,"area":null},{"id":"2508.03105","title":"Accelerating SGDM via Learning Rate and Batch Size Schedules: A Lyapunov-Based Analysis","created":"2025-08-05","cited_by_count":1,"area":null},{"id":"2508.06743","title":"Analysis of Schedule-Free Nonconvex Optimization","created":"2025-08-08","cited_by_count":1,"area":null},{"id":"2510.04988","title":"Adaptive Memory Momentum via a Model-Based Framework for Deep Learning Optimization","created":"2025-10-06","cited_by_count":1,"area":null},{"id":"2602.05657","title":"Tight Long-Term Tail Decay of (Clipped) SGD in Non-Convex Optimization","created":"2026-02-05","cited_by_count":1,"area":null},{"id":"2603.16042","title":"Shuffling the Stochastic Mirror Descent via Dual Lipschitz Continuity and Kernel Conditioning","created":"2026-03-17","cited_by_count":1,"area":null},{"id":"2606.19302","title":"Optimal scenario design for climate emulation","created":"2026-06-17","cited_by_count":1,"area":null},{"id":"2607.04233","title":"Unified convergence analysis for gradient descent optimization methods in the training of deep neural networks","created":"2026-07-05","cited_by_count":1,"area":null},{"id":"2607.08104","title":"Vanilla SGD with Momentum Survives Heavy-Tailed Noise: Convergence Analysis without Gradient Clipping or Normalization","created":"2026-07-09","cited_by_count":1,"area":null},{"id":"2208.02208","title":"SGEM: stochastic gradient with energy and momentum","created":"2022-08-03","cited_by_count":0,"area":null},{"id":"2303.14999","title":"Transformer-based Multi-Instance Learning for Weakly Supervised Object Detection","created":"2023-03-27","cited_by_count":0,"area":null},{"id":"2305.07091","title":"Stability and Convergence of Distributed Stochastic Approximations with large Unbounded Stochastic Information Delays","created":"2023-05-11","cited_by_count":0,"area":null},{"id":"2308.11220","title":"Federated Learning on Patient Data for Privacy-Protecting Polycystic Ovary Syndrome Treatment","created":"2023-08-22","cited_by_count":0,"area":null},{"id":"2310.05692","title":"Based on What We Can Control Artificial Neural Networks","created":"2023-10-09","cited_by_count":0,"area":null},{"id":"2402.00389","title":"On the $O(\\frac{\\sqrt{d}}{T^{1/4}})$ Convergence Rate of RMSProp and Its Momentum Extension Measured by $\\ell_1$ Norm","created":"2024-02-01","cited_by_count":0,"area":null},{"id":"2402.11427","title":"OptEx: Expediting First-Order Optimization with Approximately Parallelized Iterations","created":"2024-02-18","cited_by_count":0,"area":null},{"id":"2403.11163","title":"A Selective Review on Statistical Methods for Massive Data Computation: Distributed Computing, Subsampling, and Minibatch Techniques","created":"2024-03-17","cited_by_count":0,"area":null},{"id":"2403.17833","title":"GPFL: A Gradient Projection-Based Client Selection Framework for Efficient Federated Learning","created":"2024-03-26","cited_by_count":0,"area":null},{"id":"2404.06549","title":"Variational Stochastic Gradient Descent for Deep Neural Networks","created":"2024-04-09","cited_by_count":0,"area":null},{"id":"2404.16917","title":"Grad Queue : A probabilistic framework to reinforce sparse gradients","created":"2024-04-25","cited_by_count":0,"area":null},{"id":"2405.13765","title":"On the stability of gradient descent with second order dynamics for time-varying cost functions","created":"2024-05-22","cited_by_count":0,"area":null},{"id":"2407.14618","title":"SOREL: A Stochastic Algorithm for Spectral Risks Minimization","created":"2024-07-19","cited_by_count":0,"area":null},{"id":"2410.02260","title":"FedScalar: Federated Learning with Scalar Communication for Bandwidth-Constrained Networks","created":"2024-10-03","cited_by_count":0,"area":null},{"id":"2410.15206","title":"Unsupervised Domain Adaptation Approaches for Chessboard Recognition","created":"2024-10-19","cited_by_count":0,"area":null}],"next_cursor":"100"}