{"total":126,"rows":[{"id":"2402.16788","title":"Why Transformers Need Adam: A Hessian Perspective","created":"2024-02-26","cited_by_count":124,"area":null},{"id":"2208.09632","title":"Adam Can Converge Without Any Modification On Update Rules","created":"2022-08-20","cited_by_count":114,"area":null},{"id":"2304.13972","title":"Convergence of Adam Under Relaxed Assumptions","created":"2023-04-27","cited_by_count":99,"area":null},{"id":"2103.04715","title":"Bayesian imaging using Plug & Play priors: when Langevin meets Tweedie","created":"2021-03-08","cited_by_count":97,"area":null},{"id":"2208.11195","title":"Robustness to Unbounded Smoothness of Generalized SignSGD","created":"2022-08-23","cited_by_count":81,"area":null},{"id":"2404.04454","title":"Implicit Bias of AdamW: $\\ell_\\infty$ Norm Constrained Optimization","created":"2024-04-05","cited_by_count":58,"area":null},{"id":"2304.09871","title":"A Theory on Adam Instability in Large-Scale Machine Learning","created":"2023-04-19","cited_by_count":56,"area":null},{"id":"2112.03459","title":"A Novel Convergence Analysis for Algorithms of the Adam Family","created":"2021-12-07","cited_by_count":50,"area":null},{"id":"2509.26030","title":"Muon Outperforms Adam in Tail-End Associative Memory Learning","created":"2025-09-30","cited_by_count":46,"area":null},{"id":"2301.09819","title":"Model Agnostic Sample Reweighting for Out-of-Distribution Learning","created":"2023-01-24","cited_by_count":44,"area":null},{"id":"2204.02833","title":"High Probability Bounds for a Class of Nonconvex Algorithms with AdaGrad Stepsize","created":"2022-04-06","cited_by_count":40,"area":null},{"id":"2310.17998","title":"Closing the Gap Between the Upper Bound and the Lower Bound of Adam's Iteration Complexity","created":"2023-10-27","cited_by_count":38,"area":null},{"id":"2301.12832","title":"Deep networks for system identification: a Survey","created":"2023-01-30","cited_by_count":37,"area":null},{"id":"2601.19156","title":"Convergence of Muon with Newton-Schulz","created":"2026-01-27","cited_by_count":36,"area":null},{"id":"2107.13171","title":"Learning with Multiclass AUC: Theory and Algorithms","created":"2021-07-28","cited_by_count":35,"area":null},{"id":"2211.15055","title":"AdaTask: A Task-aware Adaptive Learning Rate Approach to Multi-task Learning","created":"2022-11-28","cited_by_count":31,"area":null},{"id":"2208.09900","title":"Provable Adaptivity of Adam under Non-uniform Smoothness","created":"2022-08-21","cited_by_count":30,"area":null},{"id":"2101.11075","title":"Adaptivity without Compromise: A Momentumized, Adaptive, Dual Averaged Gradient Method for Stochastic Optimization","created":"2021-01-26","cited_by_count":28,"area":null},{"id":"2301.09880","title":"Probabilistic Bilevel Coreset Selection","created":"2023-01-24","cited_by_count":27,"area":null},{"id":"2402.01567","title":"Understanding Adam Optimizer via Online Learning of Updates: Adam is FTRL in Disguise","created":"2024-02-02","cited_by_count":26,"area":null},{"id":"2101.05471","title":"Towards Practical Adam: Non-Convexity, Convergence Theory, and Mini-Batch Acceleration","created":"2021-01-14","cited_by_count":25,"area":null},{"id":"2109.05109","title":"Toward Communication Efficient Adaptive Gradient Method","created":"2021-09-10","cited_by_count":25,"area":null},{"id":"2305.12475","title":"Two Sides of One Coin: the Limits of Untuned SGD and the Power of Adaptive Methods","created":"2023-05-21","cited_by_count":23,"area":null},{"id":"2308.00522","title":"Efficient Federated Learning via Local Adaptive Amended Optimizer with Linear Speedup","created":"2023-07-30","cited_by_count":23,"area":null},{"id":"2405.18199","title":"Adam with model exponential moving average is effective for nonconvex optimization","created":"2024-05-28","cited_by_count":23,"area":null},{"id":"2410.08198","title":"Adam Exploits $\\ell_\\infty$-geometry of Loss Landscape via Coordinate-wise Adaptivity","created":"2024-10-10","cited_by_count":23,"area":null},{"id":"2305.03938","title":"Adam-family Methods for Nonsmooth Optimization with Convergence Guarantees","created":"2023-05-06","cited_by_count":22,"area":null},{"id":"2303.00565","title":"AdaSAM: Boosting Sharpness-Aware Minimization with Adaptive Learning Rate and Momentum for Training Deep Neural Networks","created":"2023-03-01","cited_by_count":21,"area":null},{"id":"2402.03982","title":"On Convergence of Adam for Stochastic Optimization under Relaxed Assumptions","created":"2024-02-06","cited_by_count":21,"area":null},{"id":"2105.03588","title":"Facial Emotion Recognition: State of the Art Performance on FER2013","created":"2021-05-08","cited_by_count":20,"area":null},{"id":"2110.08689","title":"Classical-to-Quantum Transfer Learning for Spoken Command Recognition Based on Quantum Neural Networks","created":"2021-10-17","cited_by_count":19,"area":null},{"id":"2411.02853","title":"ADOPT: Modified Adam Can Converge with Any $\\beta_2$ with the Optimal Rate","created":"2024-11-05","cited_by_count":18,"area":null},{"id":"2110.01663","title":"Global Convergence and Stability of Stochastic Gradient Descent","created":"2021-10-04","cited_by_count":17,"area":null},{"id":"2202.06009","title":"Maximizing Communication Efficiency for Large-scale Training via 0/1 Adam","created":"2022-02-12","cited_by_count":15,"area":null},{"id":"2302.01250","title":"Identifying regions of importance in wall-bounded turbulence through explainable deep learning","created":"2023-02-02","cited_by_count":15,"area":null},{"id":"2402.13794","title":"Revisiting Convergence of AdaGrad with Relaxed Assumptions","created":"2024-02-21","cited_by_count":15,"area":null},{"id":"2407.08100","title":"Non-convergence of Adam and other adaptive stochastic gradient descent optimization methods for non-vanishing learning rates","created":"2024-07-11","cited_by_count":15,"area":null},{"id":"2407.21078","title":"Convergence rates for the Adam optimizer","created":"2024-07-29","cited_by_count":15,"area":null},{"id":"2201.06133","title":"On Maximum-a-Posteriori estimation with Plug & Play priors and stochastic gradient descent","created":"2022-01-16","cited_by_count":14,"area":null},{"id":"2307.11782","title":"Convergence of Adam for Non-convex Objectives: Relaxed Hyperparameters and Non-ergodic Case","created":"2023-07-20","cited_by_count":14,"area":null},{"id":"2403.15146","title":"On the Convergence of Adam under Non-uniform Smoothness: Separability from SGDM and Beyond","created":"2024-03-22","cited_by_count":14,"area":null},{"id":"2404.01436","title":"Convergence Guarantees for RMSProp and Adam in Generalized-smooth Non-convex Optimization with Affine Noise Variance","created":"2024-04-01","cited_by_count":14,"area":null},{"id":"2104.14840","title":"Unified Convergence Analysis for Adaptive Optimization with Moving Average Estimator","created":"2021-04-30","cited_by_count":13,"area":null},{"id":"2506.04805","title":"Adaptive Preconditioners Trigger Loss Spikes in Adam","created":"2025-06-05","cited_by_count":13,"area":null},{"id":"2205.14473","title":"Efficient-Adam: Communication-Efficient Distributed Adam","created":"2022-05-28","cited_by_count":11,"area":null},{"id":"2410.19964","title":"Understanding Adam Requires Better Rotation Dependent Assumptions","created":"2024-10-25","cited_by_count":11,"area":null},{"id":"2411.07724","title":"Convergence Rate Analysis of LION","created":"2024-11-12","cited_by_count":11,"area":null},{"id":"2602.16340","title":"The Implicit Bias of Adam and Muon on Smooth Homogeneous Neural Networks","created":"2026-02-18","cited_by_count":11,"area":null},{"id":"2106.06075","title":"A Decentralized Adaptive Momentum Method for Solving a Class of Min-Max Optimization Problems","created":"2021-06-10","cited_by_count":9,"area":null},{"id":"2210.02070","title":"Non-Convergence and Limit Cycles in the Adam optimizer","created":"2022-10-05","cited_by_count":9,"area":null},{"id":"2302.01128","title":"Mnemosyne: Learning to Train Transformers with Transformers","created":"2023-02-02","cited_by_count":9,"area":null},{"id":"2410.04458","title":"A Comprehensive Framework for Analyzing the Convergence of Adam: Bridging the Gap with SGD","created":"2024-10-06","cited_by_count":9,"area":null},{"id":"2102.06527","title":"Mutually exciting point process graphs for modelling dynamic networks","created":"2021-02-11","cited_by_count":8,"area":null},{"id":"2210.05607","title":"Divergence Results and Convergence of a Variance Reduced Version of ADAM","created":"2022-10-11","cited_by_count":8,"area":null},{"id":"2311.02000","title":"High Probability Convergence of Adam Under Unbounded Gradients and Affine Variance Noise","created":"2023-11-03","cited_by_count":8,"area":null},{"id":"2504.19426","title":"Sharp higher order convergence rates for the Adam optimizer","created":"2025-04-28","cited_by_count":8,"area":null},{"id":"2505.11840","title":"On the $O(\\frac{\\sqrt{d}}{K^{1/4}})$ Convergence Rate of AdamW Measured by $\\ell_1$ Norm","created":"2025-05-17","cited_by_count":8,"area":null},{"id":"2201.11204","title":"On the Convergence of mSGD and AdaGrad for Stochastic Optimization","created":"2022-01-26","cited_by_count":7,"area":null},{"id":"2409.04707","title":"Enhancing Deep Learning with Optimized Gradient Descent: Bridging Numerical Methods and Neural Network Training","created":"2024-09-07","cited_by_count":7,"area":null},{"id":"2310.14085","title":"Adaptive, Doubly Optimal No-Regret Learning in Strongly Monotone and Exp-Concave Games with Gradient Feedback","created":"2023-10-21","cited_by_count":6,"area":null},{"id":"2402.05453","title":"Mitigating Privacy Risk in Membership Inference by Convex-Concave Loss","created":"2024-02-08","cited_by_count":6,"area":"safety"},{"id":"2606.04662","title":"Why Muon Outperforms Adam: A Curvature Perspective","created":"2026-06-03","cited_by_count":6,"area":null},{"id":"2102.09804","title":"Local Convergence of Adaptive Gradient Descent Optimizers","created":"2021-02-19","cited_by_count":5,"area":null},{"id":"2205.14811","title":"Last-iterate convergence analysis of stochastic momentum methods for neural networks","created":"2022-05-30","cited_by_count":5,"area":null},{"id":"2310.08858","title":"Adam-family Methods with Decoupled Weight Decay in Deep Learning","created":"2023-10-13","cited_by_count":5,"area":null},{"id":"2109.08282","title":"AdaLoss: A computationally-efficient and provably convergent adaptive gradient method","created":"2021-09-17","cited_by_count":4,"area":null},{"id":"2110.05454","title":"Momentum Centering and Asynchronous Update for Adaptive Gradient Methods","created":"2021-10-11","cited_by_count":4,"area":null},{"id":"2206.13290","title":"Theoretical analysis of Adam using hyperparameters close to one without Lipschitz smoothness","created":"2022-06-27","cited_by_count":4,"area":null},{"id":"2411.15274","title":"Feature-interactive Siamese graph encoder-based image analysis to predict STAS from histopathology images in lung cancer","created":"2024-11-22","cited_by_count":4,"area":null},{"id":"2602.08372","title":"Dynamic Regret via Discounted-to-Dynamic Reduction with Applications to Curved Losses and Adam Optimizer","created":"2026-02-09","cited_by_count":4,"area":null},{"id":"2603.09581","title":"Towards Understanding Adam Convergence on Highly Degenerate Polynomials","created":"2026-03-10","cited_by_count":4,"area":null},{"id":"2207.07580","title":"Analysis, Characterization, Prediction and Attribution of Extreme Atmospheric Events with Machine Learning: a Review","created":"2022-06-03","cited_by_count":3,"area":null},{"id":"2404.08624","title":"Regularized Gradient Clipping Provably Trains Wide and Deep Neural Networks","created":"2024-04-12","cited_by_count":3,"area":null},{"id":"2407.19852","title":"Quantum Long Short-Term Memory for Drug Discovery","created":"2024-07-29","cited_by_count":3,"area":null},{"id":"2505.24399","title":"LightSAM: Parameter-Agnostic Sharpness-Aware Minimization","created":"2025-05-30","cited_by_count":3,"area":null},{"id":"2507.08963","title":"Stochastic Approximation with Block Coordinate Optimal Stepsizes","created":"2025-07-11","cited_by_count":3,"area":null},{"id":"2512.05442","title":"IdealTSF: Can Non-Ideal Data Contribute to Enhancing the Performance of Time Series Forecasting Models?","created":"2025-12-05","cited_by_count":3,"area":null},{"id":"2603.18899","title":"Uniform a priori bounds and error analysis for the Adam stochastic gradient descent optimization method","created":"2026-03-19","cited_by_count":3,"area":null},{"id":"2605.18694","title":"Can Adaptive Gradient Methods Converge under Heavy-Tailed Noise? A Case Study of AdaGrad","created":"2026-05-18","cited_by_count":3,"area":null},{"id":"2606.09658","title":"Muon Learns More Robust and Transferable Features than Adam","created":"2026-06-08","cited_by_count":3,"area":null},{"id":"2106.10022","title":"Local AdaGrad-Type Algorithm for Stochastic Convex-Concave Optimization","created":"2021-06-18","cited_by_count":2,"area":null},{"id":"2305.05675","title":"UAdam: Unified Adam-Type Algorithmic Framework for Non-Convex Stochastic Optimization","created":"2023-05-09","cited_by_count":2,"area":null},{"id":"2309.09719","title":"FedLALR: Client-Specific Adaptive Learning Rates Achieve Linear Speedup for Non-IID Data","created":"2023-09-18","cited_by_count":2,"area":null},{"id":"2311.08745","title":"Using Stochastic Gradient Descent to Smooth Nonconvex Functions: Analysis of Implicit Graduated Optimization","created":"2023-11-15","cited_by_count":2,"area":null},{"id":"2403.01713","title":"MCA: Moment Channel Attention Networks","created":"2024-03-04","cited_by_count":2,"area":null},{"id":"2411.07120","title":"Lean and Mean Adaptive Optimization via Subset-Norm and Subspace-Momentum with Convergence Guarantees","created":"2024-11-11","cited_by_count":2,"area":null},{"id":"2502.02727","title":"Gradient Correction in Federated Learning with Adaptive Optimization","created":"2025-02-04","cited_by_count":2,"area":null},{"id":"2502.10784","title":"Preconditioned Inexact Stochastic ADMM for Deep Model","created":"2025-02-15","cited_by_count":2,"area":null},{"id":"2604.17423","title":"A unified convergence theory for adaptive first-order methods in the nonconvex case, including AdaNorm, full and diagonal AdaGrad and Muon","created":"2026-04-19","cited_by_count":2,"area":null},{"id":"2606.17526","title":"MGUP: A Momentum-Gradient Alignment Update Policy for Stochastic Optimization","created":"2026-06-16","cited_by_count":2,"area":null},{"id":"2208.09814","title":"Critical Bach Size Minimizes Stochastic First-Order Oracle Complexity of Deep Learning Optimizer using Hyperparameters Close to One","created":"2022-08-21","cited_by_count":1,"area":null},{"id":"2212.05796","title":"Generalizing DP-SGD with Shuffling and Batch Clipping","created":"2022-12-12","cited_by_count":1,"area":null},{"id":"2301.08442","title":"Revisiting Estimation Bias in Policy Gradients for Deep Reinforcement Learning","created":"2023-01-20","cited_by_count":1,"area":null},{"id":"2406.02939","title":"Achieving Near-Optimal Convergence for Distributed Minimax Optimization with Adaptive Stepsizes","created":"2024-06-05","cited_by_count":1,"area":null},{"id":"2408.04819","title":"Interventional Causal Structure Discovery over Graphical Models with Convergence and Optimality Guarantees","created":"2024-08-09","cited_by_count":1,"area":null},{"id":"2505.24527","title":"Optimal Density Functions for Weighted Convolution in Learning Models","created":"2025-05-30","cited_by_count":1,"area":null},{"id":"2506.21129","title":"Curriculum-Adapted Robust Reinforcement Learning for UAV Deconfliction in Adversarial Environments","created":"2025-06-26","cited_by_count":1,"area":"deeprl"},{"id":"2512.06795","title":"ADAM Optimization with Adaptive Batch Selection","created":"2025-12-07","cited_by_count":1,"area":null},{"id":"2512.07863","title":"SetAD: Semi-Supervised Anomaly Learning in Contextual Sets","created":"2025-11-26","cited_by_count":1,"area":null},{"id":"2603.02649","title":"HomeAdam: Adam and AdamW Algorithms Sometimes Go Home to Obtain Better Provable Generalization","created":"2026-03-03","cited_by_count":1,"area":null}],"next_cursor":"100"}