{"total":178,"rows":[{"id":"2208.06677","title":"Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Models","created":"2022-08-13","cited_by_count":174,"area":null},{"id":"2203.08065","title":"Surrogate Gap Minimization Improves Sharpness-Aware Training","created":"2022-03-15","cited_by_count":172,"area":null},{"id":"2402.16788","title":"Why Transformers Need Adam: A Hessian Perspective","created":"2024-02-26","cited_by_count":124,"area":null},{"id":"2212.01197","title":"FedALA: Adaptive Local Aggregation for Personalized Federated Learning","created":"2022-12-02","cited_by_count":122,"area":null},{"id":"2208.09632","title":"Adam Can Converge Without Any Modification On Update Rules","created":"2022-08-20","cited_by_count":114,"area":null},{"id":"2205.10287","title":"On the SDEs and Scaling Rules for Adaptive Gradient Algorithms","created":"2022-05-20","cited_by_count":104,"area":null},{"id":"2304.13972","title":"Convergence of Adam Under Relaxed Assumptions","created":"2023-04-27","cited_by_count":99,"area":null},{"id":"2102.02888","title":"1-bit Adam: Communication Efficient Large-Scale Training with Adam's Convergence Speed","created":"2021-02-04","cited_by_count":75,"area":null},{"id":"2404.04454","title":"Implicit Bias of AdamW: $\\ell_\\infty$ Norm Constrained Optimization","created":"2024-04-05","cited_by_count":58,"area":null},{"id":"2303.03108","title":"Gradient Norm Aware Minimization Seeks First-Order Flatness and Improves Generalization","created":"2023-03-03","cited_by_count":57,"area":null},{"id":"2205.02719","title":"Communication-Efficient Adaptive Federated Learning","created":"2022-05-05","cited_by_count":55,"area":null},{"id":"2112.03459","title":"A Novel Convergence Analysis for Algorithms of the Adam Family","created":"2021-12-07","cited_by_count":50,"area":null},{"id":"2108.11371","title":"Understanding the Generalization of Adam in Learning Neural Networks with Proper Regularization","created":"2021-08-25","cited_by_count":49,"area":null},{"id":"2106.11309","title":"How Do Adam and Training Strategies Help BNNs Optimization?","created":"2021-06-21","cited_by_count":46,"area":null},{"id":"2202.00089","title":"Understanding AdamW through Proximal Methods and Scale-Freeness","created":"2022-01-31","cited_by_count":46,"area":null},{"id":"2106.08208","title":"SUPER-ADAM: Faster and Universal Framework of Adaptive Gradients","created":"2021-06-15","cited_by_count":40,"area":null},{"id":"2204.02833","title":"High Probability Bounds for a Class of Nonconvex Algorithms with AdaGrad Stepsize","created":"2022-04-06","cited_by_count":40,"area":null},{"id":"2201.08025","title":"Low-Pass Filtering SGD for Recovering Flat Optima in the Deep Learning Optimization Landscape","created":"2022-01-20","cited_by_count":39,"area":null},{"id":"2310.17998","title":"Closing the Gap Between the Upper Bound and the Lower Bound of Adam's Iteration Complexity","created":"2023-10-27","cited_by_count":38,"area":null},{"id":"2101.03244","title":"End-to-end Prostate Cancer Detection in bpMRI via 3D CNNs: Effects of Attention Mechanisms, Clinical Priori and Decoupled False Positive Reduction","created":"2021-01-08","cited_by_count":32,"area":null},{"id":"2509.15816","title":"On the Convergence of Muon and Beyond","created":"2025-09-19","cited_by_count":31,"area":null},{"id":"2208.09900","title":"Provable Adaptivity of Adam under Non-uniform Smoothness","created":"2022-08-21","cited_by_count":30,"area":null},{"id":"2103.17182","title":"Positive-Negative Momentum: Manipulating Stochastic Gradient Noise to Improve Generalization","created":"2021-03-31","cited_by_count":28,"area":null},{"id":"2307.11108","title":"Flatness-Aware Minimization for Domain Generalization","created":"2023-07-20","cited_by_count":27,"area":null},{"id":"2101.05471","title":"Towards Practical Adam: Non-Convexity, Convergence Theory, and Mini-Batch Acceleration","created":"2021-01-14","cited_by_count":25,"area":null},{"id":"2109.05109","title":"Toward Communication Efficient Adaptive Gradient Method","created":"2021-09-10","cited_by_count":25,"area":null},{"id":"2305.07583","title":"MoMo: Momentum Models for Adaptive Learning Rates","created":"2023-05-12","cited_by_count":24,"area":null},{"id":"2305.12475","title":"Two Sides of One Coin: the Limits of Untuned SGD and the Power of Adaptive Methods","created":"2023-05-21","cited_by_count":23,"area":null},{"id":"2405.12807","title":"FAdam: Adam is a natural gradient optimizer using diagonal empirical Fisher information","created":"2024-05-21","cited_by_count":18,"area":null},{"id":"2109.03194","title":"On the Convergence of Decentralized Adaptive Gradient Methods","created":"2021-09-07","cited_by_count":17,"area":null},{"id":"2202.07530","title":"Optimal Algorithms for Stochastic Multi-Level Compositional Optimization","created":"2022-02-15","cited_by_count":17,"area":null},{"id":"2201.13011","title":"On the Power-Law Hessian Spectrums in Deep Learning","created":"2022-01-31","cited_by_count":16,"area":null},{"id":"2202.06009","title":"Maximizing Communication Efficiency for Large-scale Training via 0/1 Adam","created":"2022-02-12","cited_by_count":15,"area":null},{"id":"2205.08515","title":"Unsupervised Segmentation in Real-World Images via Spelke Object Inference","created":"2022-05-17","cited_by_count":15,"area":null},{"id":"2306.12747","title":"Don't be so Monotone: Relaxing Stochastic Line Search in Over-Parameterized Models","created":"2023-06-22","cited_by_count":15,"area":null},{"id":"2107.01173","title":"Momentum Accelerates the Convergence of Stochastic AUPRC Maximization","created":"2021-07-02","cited_by_count":14,"area":null},{"id":"2307.11782","title":"Convergence of Adam for Non-convex Objectives: Relaxed Hyperparameters and Non-ergodic Case","created":"2023-07-20","cited_by_count":14,"area":null},{"id":"2405.16397","title":"AdaFisher: Adaptive Second Order Optimization via Fisher Information","created":"2024-05-26","cited_by_count":14,"area":null},{"id":"2405.20763","title":"Improving Generalization and Convergence by Enhancing Implicit Regularization","created":"2024-05-31","cited_by_count":14,"area":null},{"id":"2104.14840","title":"Unified Convergence Analysis for Adaptive Optimization with Moving Average Estimator","created":"2021-04-30","cited_by_count":13,"area":null},{"id":"2102.04271","title":"Curse of Dimensionality for TSK Fuzzy Neural Networks: Explanation and Solutions","created":"2021-02-08","cited_by_count":12,"area":null},{"id":"2104.09808","title":"Measuring the Ripeness of Fruit with Hyperspectral Imaging and Deep Learning","created":"2021-04-20","cited_by_count":12,"area":null},{"id":"2111.00705","title":"Communication-Compressed Adaptive Gradient Method for Distributed Nonconvex Optimization","created":"2021-11-01","cited_by_count":12,"area":null},{"id":"2105.13937","title":"Polygonal Unadjusted Langevin Algorithms: Creating stable and efficient adaptive algorithms for neural networks","created":"2021-05-28","cited_by_count":11,"area":null},{"id":"2201.06714","title":"AdaTerm: Adaptive T-Distribution Estimated Robust Moments for Noise-Robust Stochastic Gradient Optimization","created":"2022-01-18","cited_by_count":11,"area":null},{"id":"2102.07002","title":"On the Last Iterate Convergence of Momentum Methods","created":"2021-02-13","cited_by_count":10,"area":null},{"id":"2109.14213","title":"On the One-sided Convergence of Adam-type Algorithms in Non-convex Non-concave Min-max Optimization","created":"2021-09-29","cited_by_count":9,"area":null},{"id":"2310.11291","title":"An Automatic Learning Rate Schedule Algorithm for Achieving Faster Convergence and Steeper Descent","created":"2023-10-17","cited_by_count":9,"area":null},{"id":"2110.09057","title":"Training Deep Neural Networks with Adaptive Momentum Inspired by the Quadratic Optimization","created":"2021-10-18","cited_by_count":8,"area":null},{"id":"2210.11639","title":"HesScale: Scalable Computation of Hessian Diagonals","created":"2022-10-20","cited_by_count":8,"area":null},{"id":"2302.00195","title":"Weight Prediction Boosts the Convergence of AdamW","created":"2023-02-01","cited_by_count":8,"area":null},{"id":"2411.19671","title":"On the Performance Analysis of Momentum Method: A Frequency Domain Perspective","created":"2024-11-29","cited_by_count":8,"area":null},{"id":"2505.11840","title":"On the $O(\\frac{\\sqrt{d}}{K^{1/4}})$ Convergence Rate of AdamW Measured by $\\ell_1$ Norm","created":"2025-05-17","cited_by_count":8,"area":null},{"id":"2107.01832","title":"Provable Convergence of Nesterov's Accelerated Gradient Method for Over-Parameterized Neural Networks","created":"2021-07-05","cited_by_count":7,"area":null},{"id":"2201.09196","title":"Learning to Predict Gradients for Semi-Supervised Continual Learning","created":"2022-01-23","cited_by_count":7,"area":null},{"id":"2306.14522","title":"Nonconvex Stochastic Bregman Proximal Gradient Method with Application to Deep Learning","created":"2023-06-26","cited_by_count":7,"area":null},{"id":"2106.11514","title":"Rethinking Adam: A Twofold Exponential Moving Average Approach","created":"2021-06-22","cited_by_count":6,"area":null},{"id":"2107.03331","title":"KOALA: A Kalman Optimization Algorithm with Loss Adaptivity","created":"2021-07-07","cited_by_count":6,"area":null},{"id":"2307.00631","title":"Bidirectional Looking with A Novel Double Exponential Moving Average to Adaptive and Non-adaptive Momentum Optimizers","created":"2023-07-02","cited_by_count":6,"area":null},{"id":"2401.08893","title":"MADA: Meta-Adaptive Optimizers through hyper-gradient Descent","created":"2024-01-17","cited_by_count":6,"area":null},{"id":"2102.09804","title":"Local Convergence of Adaptive Gradient Descent Optimizers","created":"2021-02-19","cited_by_count":5,"area":null},{"id":"2205.14811","title":"Last-iterate convergence analysis of stochastic momentum methods for neural networks","created":"2022-05-30","cited_by_count":5,"area":null},{"id":"2210.06364","title":"AdaNorm: Adaptive Gradient Norm Correction based Optimizer for CNNs","created":"2022-10-12","cited_by_count":5,"area":null},{"id":"2211.03970","title":"On the Algorithmic Stability and Generalization of Adaptive Optimization Methods","created":"2022-11-08","cited_by_count":5,"area":null},{"id":"2305.18240","title":"XGrad: Boosting Gradient-Based Optimizers With Weight Prediction","created":"2023-05-26","cited_by_count":5,"area":null},{"id":"2310.08858","title":"Adam-family Methods with Decoupled Weight Decay in Deep Learning","created":"2023-10-13","cited_by_count":5,"area":null},{"id":"2410.05140","title":"Tuning-Free Bilevel Optimization: New Algorithms and Convergence Analysis","created":"2024-10-07","cited_by_count":5,"area":null},{"id":"2102.07500","title":"On the Impact of Device and Behavioral Heterogeneity in Federated Learning","created":"2021-02-15","cited_by_count":4,"area":null},{"id":"2106.02203","title":"Adam in Private: Secure and Fast Training of Deep Neural Networks with Adaptive Moment Estimation","created":"2021-06-04","cited_by_count":4,"area":null},{"id":"2110.05454","title":"Momentum Centering and Asynchronous Update for Adaptive Gradient Methods","created":"2021-10-11","cited_by_count":4,"area":null},{"id":"2204.08306","title":"A Convergence Analysis of Nesterov's Accelerated Gradient Method in Training Deep Linear Neural Networks","created":"2022-04-18","cited_by_count":4,"area":null},{"id":"2206.13290","title":"Theoretical analysis of Adam using hyperparameters close to one without Lipschitz smoothness","created":"2022-06-27","cited_by_count":4,"area":null},{"id":"2304.05635","title":"Unifying and Personalizing Weakly-supervised Federated Medical Image Segmentation via Adaptive Representation and Aggregation","created":"2023-04-12","cited_by_count":4,"area":null},{"id":"2305.18666","title":"BiSLS/SPS: Auto-tune Step Sizes for Stable Bi-level Optimization","created":"2023-05-30","cited_by_count":4,"area":null},{"id":"2312.01658","title":"AGD: an Auto-switchable Optimizer using Stepwise Gradient Difference for Preconditioning Matrix","created":"2023-12-04","cited_by_count":4,"area":null},{"id":"2602.08372","title":"Dynamic Regret via Discounted-to-Dynamic Reduction with Applications to Curved Losses and Adam Optimizer","created":"2026-02-09","cited_by_count":4,"area":null},{"id":"2101.01847","title":"Deep Learning for Fast and Reliable Initial Access in AI-Driven 6G mmWave Networks","created":"2021-01-06","cited_by_count":3,"area":null},{"id":"2201.12678","title":"A Stochastic Bundle Method for Interpolating Networks","created":"2022-01-29","cited_by_count":3,"area":null},{"id":"2203.12191","title":"An Adaptive Gradient Method with Energy and Momentum","created":"2022-03-23","cited_by_count":3,"area":null},{"id":"2208.08780","title":"GraVoS: Voxel Selection for 3D Point-Cloud Detection","created":"2022-08-18","cited_by_count":3,"area":null},{"id":"2302.01029","title":"On Suppressing Range of Adaptive Stepsizes of Adam to Improve Generalisation Performance","created":"2023-02-02","cited_by_count":3,"area":null},{"id":"2410.06369","title":"Communication-Efficient Federated Group Distributionally Robust Optimization","created":"2024-10-08","cited_by_count":3,"area":null},{"id":"2410.06373","title":"Unveiling the Backbone-Optimizer Coupling Bias in Visual Representation Learning","created":"2024-10-08","cited_by_count":3,"area":null},{"id":"2510.27153","title":"Exploring Landscapes for Better Minima along Valleys","created":"2025-10-31","cited_by_count":3,"area":null},{"id":"2109.12504","title":"AdaInject: Injection Based Adaptive Gradient Descent Optimizers for Convolutional Neural Networks","created":"2021-09-26","cited_by_count":2,"area":null},{"id":"2110.14794","title":"Masked LARk: Masked Learning, Aggregation and Reporting worKflow","created":"2021-10-27","cited_by_count":2,"area":null},{"id":"2203.16723","title":"Exploiting Explainable Metrics for Augmented SGD","created":"2022-03-31","cited_by_count":2,"area":null},{"id":"2302.06103","title":"FedDA: Faster Framework of Local Adaptive Gradient Methods via Restarted Dual Averaging","created":"2023-02-13","cited_by_count":2,"area":null},{"id":"2305.05675","title":"UAdam: Unified Adam-Type Algorithmic Framework for Non-Convex Stochastic Optimization","created":"2023-05-09","cited_by_count":2,"area":null},{"id":"2308.09604","title":"Faster Stochastic Variance Reduction Methods for Compositional MiniMax Optimization","created":"2023-08-18","cited_by_count":2,"area":null},{"id":"2312.12606","title":"Optimizing Neural Networks with Gradient Lexicase Selection","created":"2023-12-19","cited_by_count":2,"area":null},{"id":"2412.12473","title":"A Method for Enhancing Generalization of Adam by Multiple Integrations","created":"2024-12-17","cited_by_count":2,"area":null},{"id":"2412.20302","title":"EXAdam: The Power of Adaptive Cross-Moments","created":"2024-12-29","cited_by_count":2,"area":null},{"id":"2502.10784","title":"Preconditioned Inexact Stochastic ADMM for Deep Model","created":"2025-02-15","cited_by_count":2,"area":null},{"id":"2502.12298","title":"Symmetric Rank-One Quasi-Newton Methods for Deep Learning Using Cubic Regularization","created":"2025-02-17","cited_by_count":2,"area":null},{"id":"2506.22479","title":"Hindsight-Guided Momentum (HGM) Optimizer: An Approach to Adaptive Learning Rate","created":"2025-06-22","cited_by_count":2,"area":null},{"id":"2512.06370","title":"Greedy Alignment Principle for Optimizer Selection","created":"2025-12-06","cited_by_count":2,"area":null},{"id":"2603.01040","title":"Fed-ADE: Adaptive Learning Rate for Federated Post-adaptation under Distribution Shift","created":"2026-03-01","cited_by_count":2,"area":null},{"id":"2604.08939","title":"Delve into the Applicability of Advanced Optimizers for Multi-Task Learning","created":"2026-04-10","cited_by_count":2,"area":null},{"id":"2605.00650","title":"AdaMeZO: Adam-style Zeroth-Order Optimizer for LLM Fine-tuning Without Maintaining the Moments","created":"2026-05-01","cited_by_count":2,"area":null}],"next_cursor":"100"}