{"total":91,"rows":[{"id":"2208.06677","title":"Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Models","created":"2022-08-13","cited_by_count":174,"area":null},{"id":"2402.16788","title":"Why Transformers Need Adam: A Hessian Perspective","created":"2024-02-26","cited_by_count":124,"area":null},{"id":"2208.09632","title":"Adam Can Converge Without Any Modification On Update Rules","created":"2022-08-20","cited_by_count":114,"area":null},{"id":"2207.14484","title":"Adaptive Gradient Methods at the Edge of Stability","created":"2022-07-29","cited_by_count":101,"area":null},{"id":"2304.13972","title":"Convergence of Adam Under Relaxed Assumptions","created":"2023-04-27","cited_by_count":99,"area":null},{"id":"2202.05791","title":"The Power of Adaptivity in SGD: Self-Tuning Step Sizes with Unbounded Gradients and Affine Variance","created":"2022-02-11","cited_by_count":61,"area":null},{"id":"2205.02719","title":"Communication-Efficient Adaptive Federated Learning","created":"2022-05-05","cited_by_count":55,"area":null},{"id":"2306.00204","title":"Toward Understanding Why Adam Converges Faster Than SGD for Transformers","created":"2023-05-31","cited_by_count":53,"area":null},{"id":"2202.00089","title":"Understanding AdamW through Proximal Methods and Scale-Freeness","created":"2022-01-31","cited_by_count":46,"area":null},{"id":"2509.26030","title":"Muon Outperforms Adam in Tail-End Associative Memory Learning","created":"2025-09-30","cited_by_count":46,"area":null},{"id":"2305.08048","title":"Towards Understanding the Generalization of Graph Neural Networks","created":"2023-05-14","cited_by_count":41,"area":null},{"id":"2204.02833","title":"High Probability Bounds for a Class of Nonconvex Algorithms with AdaGrad Stepsize","created":"2022-04-06","cited_by_count":40,"area":null},{"id":"2410.16103","title":"LDAdam: Adaptive Optimization from Low-Dimensional Gradient Statistics","created":"2024-10-21","cited_by_count":38,"area":null},{"id":"2202.04428","title":"Adapting to Mixing Time in Stochastic Optimization with Markovian Data","created":"2022-02-09","cited_by_count":36,"area":null},{"id":"2406.10650","title":"The Implicit Bias of Adam on Separable Data","created":"2024-06-15","cited_by_count":36,"area":null},{"id":"2509.15816","title":"On the Convergence of Muon and Beyond","created":"2025-09-19","cited_by_count":31,"area":null},{"id":"2208.09900","title":"Provable Adaptivity of Adam under Non-uniform Smoothness","created":"2022-08-21","cited_by_count":30,"area":null},{"id":"2302.08783","title":"SGD with AdaGrad Stepsizes: Full Adaptivity with High Probability to Unknown Parameters, Unbounded Gradients and Affine Variance","created":"2023-02-17","cited_by_count":30,"area":null},{"id":"2101.11075","title":"Adaptivity without Compromise: A Momentumized, Adaptive, Dual Averaged Gradient Method for Stochastic Optimization","created":"2021-01-26","cited_by_count":28,"area":null},{"id":"2101.05471","title":"Towards Practical Adam: Non-Convexity, Convergence Theory, and Mini-Batch Acceleration","created":"2021-01-14","cited_by_count":25,"area":null},{"id":"2206.07082","title":"Stability and Generalization of Stochastic Optimization with Nonconvex and Nonsmooth Problems","created":"2022-06-14","cited_by_count":25,"area":null},{"id":"2305.12475","title":"Two Sides of One Coin: the Limits of Untuned SGD and the Power of Adaptive Methods","created":"2023-05-21","cited_by_count":23,"area":null},{"id":"2308.00522","title":"Efficient Federated Learning via Local Adaptive Amended Optimizer with Linear Speedup","created":"2023-07-30","cited_by_count":23,"area":null},{"id":"2410.08198","title":"Adam Exploits $\\ell_\\infty$-geometry of Loss Landscape via Coordinate-wise Adaptivity","created":"2024-10-10","cited_by_count":23,"area":null},{"id":"2303.00565","title":"AdaSAM: Boosting Sharpness-Aware Minimization with Adaptive Learning Rate and Momentum for Training Deep Neural Networks","created":"2023-03-01","cited_by_count":21,"area":null},{"id":"2402.03982","title":"On Convergence of Adam for Stochastic Optimization under Relaxed Assumptions","created":"2024-02-06","cited_by_count":21,"area":null},{"id":"2205.05632","title":"On Distributed Adaptive Optimization with Gradient Compression","created":"2022-05-11","cited_by_count":19,"area":null},{"id":"2402.07114","title":"Towards Quantifying the Preconditioning Effect of Adam","created":"2024-02-11","cited_by_count":19,"area":null},{"id":"2405.15593","title":"MicroAdam: Accurate Adaptive Optimization with Low Space Overhead and Provable Convergence","created":"2024-05-24","cited_by_count":19,"area":null},{"id":"2411.02853","title":"ADOPT: Modified Adam Can Converge with Any $\\beta_2$ with the Optimal Rate","created":"2024-11-05","cited_by_count":18,"area":null},{"id":"2109.03194","title":"On the Convergence of Decentralized Adaptive Gradient Methods","created":"2021-09-07","cited_by_count":17,"area":null},{"id":"2202.06009","title":"Maximizing Communication Efficiency for Large-scale Training via 0/1 Adam","created":"2022-02-12","cited_by_count":15,"area":null},{"id":"2402.13794","title":"Revisiting Convergence of AdaGrad with Relaxed Assumptions","created":"2024-02-21","cited_by_count":15,"area":null},{"id":"2307.11782","title":"Convergence of Adam for Non-convex Objectives: Relaxed Hyperparameters and Non-ergodic Case","created":"2023-07-20","cited_by_count":14,"area":null},{"id":"2506.04805","title":"Adaptive Preconditioners Trigger Loss Spikes in Adam","created":"2025-06-05","cited_by_count":13,"area":null},{"id":"2111.00705","title":"Communication-Compressed Adaptive Gradient Method for Distributed Nonconvex Optimization","created":"2021-11-01","cited_by_count":12,"area":null},{"id":"2410.19964","title":"Understanding Adam Requires Better Rotation Dependent Assumptions","created":"2024-10-25","cited_by_count":11,"area":null},{"id":"2109.14213","title":"On the One-sided Convergence of Adam-type Algorithms in Non-convex Non-concave Min-max Optimization","created":"2021-09-29","cited_by_count":9,"area":null},{"id":"2202.04296","title":"A Projection-free Algorithm for Constrained Stochastic Multi-level Composition Optimization","created":"2022-02-09","cited_by_count":8,"area":null},{"id":"2210.17478","title":"TiAda: A Time-scale Adaptive Algorithm for Nonconvex Minimax Optimization","created":"2022-10-31","cited_by_count":8,"area":null},{"id":"2311.02000","title":"High Probability Convergence of Adam Under Unbounded Gradients and Affine Variance Noise","created":"2023-11-03","cited_by_count":8,"area":null},{"id":"2402.11215","title":"AdAdaGrad: Adaptive Batch Size Schemes for Adaptive Gradient Methods","created":"2024-02-17","cited_by_count":8,"area":null},{"id":"2403.04236","title":"Regularized DeepIV with Model Selection","created":"2024-03-07","cited_by_count":7,"area":null},{"id":"2110.04844","title":"Frequency-aware SGD for Efficient Embedding Learning with Provable Benefits","created":"2021-10-10","cited_by_count":6,"area":null},{"id":"2302.07049","title":"Multilevel Objective-Function-Free Optimization with an Application to Neural Networks Training","created":"2023-02-14","cited_by_count":6,"area":null},{"id":"2606.04662","title":"Why Muon Outperforms Adam: A Curvature Perspective","created":"2026-06-03","cited_by_count":6,"area":null},{"id":"2201.05500","title":"Communication-Efficient TeraByte-Scale Model Training Framework for Online Advertising","created":"2022-01-05","cited_by_count":5,"area":null},{"id":"2211.03970","title":"On the Algorithmic Stability and Generalization of Adaptive Optimization Methods","created":"2022-11-08","cited_by_count":5,"area":null},{"id":"2307.13680","title":"High Probability Analysis for Non-Convex Stochastic Optimization with Clipping","created":"2023-07-25","cited_by_count":5,"area":null},{"id":"2506.01863","title":"Unified Scaling Laws for Compressed Representations","created":"2025-06-02","cited_by_count":5,"area":null},{"id":"2108.09134","title":"Accelerating Federated Learning with a Global Biased Optimiser","created":"2021-08-20","cited_by_count":4,"area":null},{"id":"2109.08282","title":"AdaLoss: A computationally-efficient and provably convergent adaptive gradient method","created":"2021-09-17","cited_by_count":4,"area":null},{"id":"2206.13290","title":"Theoretical analysis of Adam using hyperparameters close to one without Lipschitz smoothness","created":"2022-06-27","cited_by_count":4,"area":null},{"id":"2312.01658","title":"AGD: an Auto-switchable Optimizer using Stepwise Gradient Difference for Preconditioning Matrix","created":"2023-12-04","cited_by_count":4,"area":null},{"id":"2410.00232","title":"Preconditioning for Accelerated Gradient Descent Optimization and Regularization","created":"2024-09-30","cited_by_count":4,"area":null},{"id":"2411.05746","title":"Continuous-Time Analysis of Adaptive Optimization and Normalization","created":"2024-11-08","cited_by_count":4,"area":null},{"id":"2603.09581","title":"Towards Understanding Adam Convergence on Highly Degenerate Polynomials","created":"2026-03-10","cited_by_count":4,"area":null},{"id":"2110.00532","title":"Layer-wise and Dimension-wise Locally Adaptive Federated Learning","created":"2021-10-01","cited_by_count":3,"area":null},{"id":"2210.04319","title":"Dissecting adaptive methods in GANs","created":"2022-10-09","cited_by_count":3,"area":null},{"id":"2211.14292","title":"Analysis of Error Feedback in Federated Non-Convex Optimization with Biased Compression","created":"2022-11-25","cited_by_count":3,"area":null},{"id":"2409.00029","title":"Attack Anything: Blind DNNs via Universal Background Adversarial Attack","created":"2024-08-17","cited_by_count":3,"area":null},{"id":"2505.24399","title":"LightSAM: Parameter-Agnostic Sharpness-Aware Minimization","created":"2025-05-30","cited_by_count":3,"area":null},{"id":"2606.09658","title":"Muon Learns More Robust and Transferable Features than Adam","created":"2026-06-08","cited_by_count":3,"area":null},{"id":"2210.16205","title":"Local Model Reconstruction Attacks in Federated Learning and their Uses","created":"2022-10-28","cited_by_count":2,"area":null},{"id":"2302.06103","title":"FedDA: Faster Framework of Local Adaptive Gradient Methods via Restarted Dual Averaging","created":"2023-02-13","cited_by_count":2,"area":null},{"id":"2305.05675","title":"UAdam: Unified Adam-Type Algorithmic Framework for Non-Convex Stochastic Optimization","created":"2023-05-09","cited_by_count":2,"area":null},{"id":"2309.09719","title":"FedLALR: Client-Specific Adaptive Learning Rates Achieve Linear Speedup for Non-IID Data","created":"2023-09-18","cited_by_count":2,"area":null},{"id":"2310.12969","title":"Demystifying the Myths and Legends of Nonconvex Convergence of SGD","created":"2023-10-19","cited_by_count":2,"area":null},{"id":"2311.08745","title":"Using Stochastic Gradient Descent to Smooth Nonconvex Functions: Analysis of Implicit Graduated Optimization","created":"2023-11-15","cited_by_count":2,"area":null},{"id":"2401.15890","title":"Probabilistic Guarantees of Stochastic Recursive Gradient in Non-Convex Finite Sum Problems","created":"2024-01-29","cited_by_count":2,"area":null},{"id":"2412.19444","title":"Towards Simple and Provable Parameter-Free Adaptive Gradient Methods","created":"2024-12-27","cited_by_count":2,"area":null},{"id":"2606.17526","title":"MGUP: A Momentum-Gradient Alignment Update Policy for Stochastic Optimization","created":"2026-06-16","cited_by_count":2,"area":null},{"id":"2101.02397","title":"A Comprehensive Study on Optimization Strategies for Gradient Descent In Deep Learning","created":"2021-01-07","cited_by_count":1,"area":null},{"id":"2208.09814","title":"Critical Bach Size Minimizes Stochastic First-Order Oracle Complexity of Deep Learning Optimizer using Hyperparameters Close to One","created":"2022-08-21","cited_by_count":1,"area":null},{"id":"2210.07454","title":"Communication-Efficient Adam-Type Algorithms for Distributed Data Mining","created":"2022-10-14","cited_by_count":1,"area":null},{"id":"2406.02939","title":"Achieving Near-Optimal Convergence for Distributed Minimax Optimization with Adaptive Stepsizes","created":"2024-06-05","cited_by_count":1,"area":null},{"id":"2411.12697","title":"Attribute Inference Attacks for Federated Regression Tasks","created":"2024-11-19","cited_by_count":1,"area":null},{"id":"2412.02153","title":"Revisiting the Initial Steps in Adaptive Gradient Descent Optimization","created":"2024-12-03","cited_by_count":1,"area":null},{"id":"2512.05084","title":"Gradient Descent with Provably Tuned Learning-rate Schedules","created":"2025-12-04","cited_by_count":1,"area":null},{"id":"2512.07112","title":"FOAM: Blocked State Folding for Memory-Efficient LLM Training","created":"2025-12-08","cited_by_count":1,"area":null},{"id":"2603.02649","title":"HomeAdam: Adam and AdamW Algorithms Sometimes Go Home to Obtain Better Provable Generalization","created":"2026-03-03","cited_by_count":1,"area":null},{"id":"2604.08742","title":"Deterministic Adam-Inspired Methods with Accelerated Convergence Rate","created":"2026-04-09","cited_by_count":1,"area":null},{"id":"2605.12878","title":"Adam-SHANG: A Convergent Adam-Type Method for Stochastic Smooth Convex Optimization","created":"2026-05-13","cited_by_count":1,"area":null},{"id":"2110.08531","title":"A theoretical and empirical study of new adaptive algorithms with additional momentum steps and shifted updates for stochastic non-convex optimization","created":"2021-10-16","cited_by_count":0,"area":null},{"id":"2208.00502","title":"Formal guarantees for heuristic optimization algorithms used in machine learning","created":"2022-07-31","cited_by_count":0,"area":null},{"id":"2309.01779","title":"DRAG: Divergence-based Adaptive Aggregation in Federated learning on Non-IID Data","created":"2023-09-04","cited_by_count":0,"area":null},{"id":"2404.01714","title":"Conjugate-Gradient-like Based Adaptive Moment Estimation Optimization Algorithm for Deep Learning","created":"2024-04-02","cited_by_count":0,"area":null},{"id":"2602.09057","title":"SVD-Preconditioned Gradient Descent Method for Solving Nonlinear Least Squares Problems","created":"2026-02-07","cited_by_count":0,"area":null},{"id":"2606.06934","title":"Uniform Stability and Generalization Error of GD and SGD on Fixed-Point Parameters","created":"2026-06-05","cited_by_count":0,"area":null},{"id":"2608.30382","title":"Convergence rates for the RMSprop optimizer with full control of the hyperparameters","created":"2026-08-31","cited_by_count":0,"area":null},{"id":"2609.30276","title":"Why Clipping Matters in AdaGrad? Toward a High-Probability Theory under Generalized Smoothness","created":"2026-08-12","cited_by_count":0,"area":null}],"next_cursor":null}