{"total":250,"rows":[{"id":"2207.14484","title":"Adaptive Gradient Methods at the Edge of Stability","created":"2022-07-29","cited_by_count":101,"area":null},{"id":"2107.04562","title":"The Bayesian Learning Rule","created":"2021-07-09","cited_by_count":99,"area":null},{"id":"2310.01082","title":"Linear attention is (maybe) all you need (to understand transformer optimization)","created":"2023-10-02","cited_by_count":98,"area":null},{"id":"2210.09221","title":"Vision Transformers provably learn spatial structure","created":"2022-10-13","cited_by_count":93,"area":null},{"id":"2310.05199","title":"Loose lips sink ships: Mitigating Length Bias in Reinforcement Learning from Human Feedback","created":"2023-10-08","cited_by_count":78,"area":"alignment"},{"id":"2505.13416","title":"Gluon: Making Muon & Scion Great Again! (Bridging Theory and Practice of LMO-based Optimizers for LLMs)","created":"2025-05-19","cited_by_count":72,"area":null},{"id":"2210.00055","title":"MaskTune: Mitigating Spurious Correlations by Forcing to Explore","created":"2022-09-30","cited_by_count":66,"area":null},{"id":"2404.04454","title":"Implicit Bias of AdamW: $\\ell_\\infty$ Norm Constrained Optimization","created":"2024-04-05","cited_by_count":58,"area":null},{"id":"2303.03108","title":"Gradient Norm Aware Minimization Seeks First-Order Flatness and Improves Generalization","created":"2023-03-03","cited_by_count":57,"area":null},{"id":"2202.05834","title":"Predicting Out-of-Distribution Error with the Projection Norm","created":"2022-02-11","cited_by_count":50,"area":null},{"id":"2108.11371","title":"Understanding the Generalization of Adam in Learning Neural Networks with Proper Regularization","created":"2021-08-25","cited_by_count":49,"area":null},{"id":"2106.11309","title":"How Do Adam and Training Strategies Help BNNs Optimization?","created":"2021-06-21","cited_by_count":46,"area":null},{"id":"2202.00089","title":"Understanding AdamW through Proximal Methods and Scale-Freeness","created":"2022-01-31","cited_by_count":46,"area":null},{"id":"2303.15725","title":"Solving Regularized Exp, Cosh and Sinh Regression Problems","created":"2023-03-28","cited_by_count":44,"area":null},{"id":"2310.20199","title":"In Search of Lost Online Test-time Adaptation: A Survey","created":"2023-10-31","cited_by_count":44,"area":null},{"id":"2106.08208","title":"SUPER-ADAM: Faster and Universal Framework of Adaptive Gradients","created":"2021-06-15","cited_by_count":40,"area":null},{"id":"2207.05931","title":"Towards understanding how momentum improves generalization in deep learning","created":"2022-07-13","cited_by_count":40,"area":null},{"id":"2505.21829","title":"In Search of Adam's Secret Sauce","created":"2025-05-27","cited_by_count":40,"area":null},{"id":"2201.08025","title":"Low-Pass Filtering SGD for Recovering Flat Optima in the Deep Learning Optimization Landscape","created":"2022-01-20","cited_by_count":39,"area":null},{"id":"2106.15324","title":"Effective Evaluation of Deep Active Learning on Image Classification Tasks","created":"2021-06-16","cited_by_count":37,"area":null},{"id":"2110.02424","title":"Spectral Bias in Practice: The Role of Function Frequency in Generalization","created":"2021-10-06","cited_by_count":37,"area":null},{"id":"2309.15639","title":"Enhancing Sharpness-Aware Optimization Through Variance Suppression","created":"2023-09-27","cited_by_count":37,"area":null},{"id":"2406.10650","title":"The Implicit Bias of Adam on Separable Data","created":"2024-06-15","cited_by_count":36,"area":null},{"id":"2305.16284","title":"DoWG Unleashed: An Efficient Universal Parameter-Free Gradient Descent Method","created":"2023-05-25","cited_by_count":35,"area":null},{"id":"2110.15210","title":"Towards Model Agnostic Federated Learning Using Knowledge Distillation","created":"2021-10-28","cited_by_count":31,"area":"distill"},{"id":"2205.12141","title":"One-Pixel Shortcut: on the Learning Preference of Deep Neural Networks","created":"2022-05-24","cited_by_count":29,"area":null},{"id":"2101.11075","title":"Adaptivity without Compromise: A Momentumized, Adaptive, Dual Averaged Gradient Method for Stochastic Optimization","created":"2021-01-26","cited_by_count":28,"area":null},{"id":"2312.14334","title":"DP-AdamBC: Your DP-Adam Is Actually DP-SGD (Unless You Apply Bias Correction)","created":"2023-12-21","cited_by_count":28,"area":null},{"id":"2307.11108","title":"Flatness-Aware Minimization for Domain Generalization","created":"2023-07-20","cited_by_count":27,"area":null},{"id":"2309.00079","title":"On the Implicit Bias of Adam","created":"2023-08-31","cited_by_count":27,"area":null},{"id":"2205.12808","title":"Mirror Descent Maximizes Generalized Margin and Can Be Implemented Efficiently","created":"2022-05-25","cited_by_count":26,"area":null},{"id":"2302.10886","title":"Some Fundamental Aspects about Lipschitz Continuity of Neural Networks","created":"2023-02-21","cited_by_count":26,"area":null},{"id":"2402.03496","title":"Can We Remove the Square-Root in Adaptive Gradient Methods? A Second-Order Perspective","created":"2024-02-05","cited_by_count":25,"area":null},{"id":"2102.07227","title":"Learning by Turning: Neural Architecture Aware Optimisation","created":"2021-02-14","cited_by_count":23,"area":null},{"id":"2211.11865","title":"Bayesian Learning for Neural Networks: an algorithmic survey","created":"2022-11-21","cited_by_count":23,"area":null},{"id":"2110.03891","title":"Does Momentum Change the Implicit Regularization on Separable Data?","created":"2021-10-08","cited_by_count":22,"area":null},{"id":"2102.03034","title":"Hyperparameter Optimization Is Deceiving Us, and How to Stop It","created":"2021-02-05","cited_by_count":20,"area":null},{"id":"2312.00359","title":"Temperature Balancing, Layer-wise Weight Analysis, and Neural Network Training","created":"2023-12-01","cited_by_count":20,"area":null},{"id":"2312.08399","title":"Principled Weight Initialization for Hypernetworks","created":"2023-12-13","cited_by_count":19,"area":null},{"id":"2410.22069","title":"Flavors of Margin: Implicit Bias of Steepest Descent in Homogeneous Neural Networks","created":"2024-10-29","cited_by_count":18,"area":null},{"id":"2102.03869","title":"Structured Sparsity Inducing Adaptive Optimizers for Deep Learning","created":"2021-02-07","cited_by_count":17,"area":null},{"id":"2208.13219","title":"Visualizing high-dimensional loss landscapes with Hessian directions","created":"2022-08-28","cited_by_count":16,"area":null},{"id":"2312.12670","title":"On the Role of Server Momentum in Federated Learning","created":"2023-12-19","cited_by_count":16,"area":null},{"id":"2402.04553","title":"Curvature-Informed SGD via General Purpose Lie-Group Preconditioners","created":"2024-02-07","cited_by_count":16,"area":null},{"id":"2407.20750","title":"JaColBERTv2.5: Optimising Multi-Vector Retrievers to Create State-of-the-Art Japanese Retrievers with Constrained Resources","created":"2024-07-30","cited_by_count":15,"area":null},{"id":"2506.12543","title":"Is your batch size the problem? Revisiting the Adam-SGD gap in language modeling","created":"2025-06-14","cited_by_count":15,"area":null},{"id":"2306.13853","title":"A Unified Approach to Controlling Implicit Regularization via Mirror Descent","created":"2023-06-24","cited_by_count":14,"area":null},{"id":"2405.16397","title":"AdaFisher: Adaptive Second Order Optimization via Fisher Information","created":"2024-05-26","cited_by_count":14,"area":null},{"id":"2407.07082","title":"Can Learned Optimization Make Reinforcement Learning Less Difficult?","created":"2024-07-09","cited_by_count":14,"area":"deeprl"},{"id":"2602.17080","title":"Adam Improves Muon: Adaptive Moment Estimation with Orthogonalized Momentum","created":"2026-02-19","cited_by_count":14,"area":null},{"id":"2505.24022","title":"The Rich and the Simple: On the Implicit Bias of Adam and SGD","created":"2025-05-29","cited_by_count":13,"area":null},{"id":"2102.04270","title":"Enabling Binary Neural Network Training on the Edge","created":"2021-02-08","cited_by_count":11,"area":null},{"id":"2105.13937","title":"Polygonal Unadjusted Langevin Algorithms: Creating stable and efficient adaptive algorithms for neural networks","created":"2021-05-28","cited_by_count":11,"area":null},{"id":"2205.14473","title":"Efficient-Adam: Communication-Efficient Distributed Adam","created":"2022-05-28","cited_by_count":11,"area":null},{"id":"2206.05985","title":"Modeling the Machine Learning Multiverse","created":"2022-06-13","cited_by_count":11,"area":null},{"id":"2206.11872","title":"Provable Acceleration of Heavy Ball beyond Quadratics for a Class of Polyak-\\L{}ojasiewicz Functions when the Non-Convexity is Averaged-Out","created":"2022-06-22","cited_by_count":11,"area":null},{"id":"2410.19964","title":"Understanding Adam Requires Better Rotation Dependent Assumptions","created":"2024-10-25","cited_by_count":11,"area":null},{"id":"2602.16340","title":"The Implicit Bias of Adam and Muon on Smooth Homogeneous Neural Networks","created":"2026-02-18","cited_by_count":11,"area":null},{"id":"2111.00856","title":"Large-Scale Deep Learning Optimizations: A Comprehensive Survey","created":"2021-11-01","cited_by_count":10,"area":null},{"id":"2203.14177","title":"Benchmarking Deep AUROC Optimization: Loss Functions and Algorithmic Choices","created":"2022-03-27","cited_by_count":10,"area":null},{"id":"2302.09738","title":"Simplifying Momentum-based Positive-definite Submanifold Optimization with Applications to Deep Learning","created":"2023-02-20","cited_by_count":10,"area":null},{"id":"2503.02225","title":"Sharpness-Aware Minimization: General Analysis and Improved Rates","created":"2025-03-04","cited_by_count":10,"area":null},{"id":"2511.04622","title":"ODE approximation for the Adam algorithm: General and overparametrized setting","created":"2025-11-06","cited_by_count":10,"area":null},{"id":"2203.12330","title":"Predicting the generalization gap in neural networks using topological data analysis","created":"2022-03-23","cited_by_count":9,"area":null},{"id":"2206.00944","title":"Feature Space Particle Inference for Neural Network Ensembles","created":"2022-06-02","cited_by_count":9,"area":null},{"id":"2206.11120","title":"Near-optimal control of dynamical systems with neural ordinary differential equations","created":"2022-06-22","cited_by_count":9,"area":null},{"id":"2401.12236","title":"The Surprising Harmfulness of Benign Overfitting for Adversarial Robustness","created":"2024-01-19","cited_by_count":9,"area":null},{"id":"2407.14340","title":"Large Kernel Distillation Network for Efficient Single Image Super-Resolution","created":"2024-07-19","cited_by_count":9,"area":null},{"id":"2106.02985","title":"Escaping Saddle Points Faster with Stochastic Momentum","created":"2021-06-05","cited_by_count":8,"area":null},{"id":"2110.09057","title":"Training Deep Neural Networks with Adaptive Momentum Inspired by the Quadratic Optimization","created":"2021-10-18","cited_by_count":8,"area":null},{"id":"2203.01400","title":"Adaptive Gradient Methods with Local Guarantees","created":"2022-03-02","cited_by_count":8,"area":null},{"id":"2206.09264","title":"Pisces: Efficient Federated Learning via Guided Asynchronous Training","created":"2022-06-18","cited_by_count":8,"area":null},{"id":"2210.11639","title":"HesScale: Scalable Computation of Hessian Diagonals","created":"2022-10-20","cited_by_count":8,"area":null},{"id":"2302.11085","title":"Learning to Generalize Provably in Learning to Optimize","created":"2023-02-22","cited_by_count":8,"area":null},{"id":"2308.06703","title":"Understanding the robustness difference between stochastic gradient descent and adaptive gradient methods","created":"2023-08-13","cited_by_count":8,"area":null},{"id":"2401.07250","title":"Stabilizing Sharpness-aware Minimization Through A Simple Renormalization Strategy","created":"2024-01-14","cited_by_count":8,"area":null},{"id":"2402.11215","title":"AdAdaGrad: Adaptive Batch Size Schemes for Adaptive Gradient Methods","created":"2024-02-17","cited_by_count":8,"area":null},{"id":"2410.13271","title":"Inductive Gradient Adjustment For Spectral Bias In Implicit Neural Representations","created":"2024-10-17","cited_by_count":8,"area":null},{"id":"2102.07314","title":"The Role of Momentum Parameters in the Optimal Convergence of Adaptive Polyak's Heavy-ball Methods","created":"2021-02-15","cited_by_count":7,"area":null},{"id":"2102.07927","title":"Structured Dropout Variational Inference for Bayesian Neural Networks","created":"2021-02-16","cited_by_count":7,"area":null},{"id":"2104.00606","title":"Model Selection's Disparate Impact in Real-World Deep Learning Applications","created":"2021-04-01","cited_by_count":7,"area":null},{"id":"2401.16687","title":"Revisiting Gradient Pruning: A Dual Realization for Defending against Gradient Attacks","created":"2024-01-30","cited_by_count":7,"area":null},{"id":"2403.12229","title":"Fusion Transformer with Object Mask Guidance for Image Forgery Analysis","created":"2024-03-18","cited_by_count":7,"area":null},{"id":"2403.17592","title":"On the Benefits of Over-parameterization for Out-of-Distribution Generalization","created":"2024-03-26","cited_by_count":7,"area":null},{"id":"2103.06733","title":"Intraclass clustering: an implicit learning ability that regularizes DNNs","created":"2021-03-11","cited_by_count":6,"area":null},{"id":"2106.11514","title":"Rethinking Adam: A Twofold Exponential Moving Average Approach","created":"2021-06-22","cited_by_count":6,"area":null},{"id":"2109.12425","title":"L$^{2}$NAS: Learning to Optimize Neural Architectures via Continuous-Action Reinforcement Learning","created":"2021-09-25","cited_by_count":6,"area":null},{"id":"2302.07851","title":"Continuized Acceleration for Quasar Convex Functions in Non-Convex Optimization","created":"2023-02-15","cited_by_count":6,"area":null},{"id":"2411.17608","title":"Mixed-State Quantum Denoising Diffusion Probabilistic Model","created":"2024-11-26","cited_by_count":6,"area":"diffusion"},{"id":"2107.11845","title":"On-Device Content Moderation","created":"2021-07-25","cited_by_count":5,"area":null},{"id":"2111.11542","title":"Depth Without the Magic: Inductive Bias of Natural Gradient Descent","created":"2021-11-22","cited_by_count":5,"area":null},{"id":"2210.16400","title":"Flatter, faster: scaling momentum for optimal speedup of SGD","created":"2022-10-28","cited_by_count":5,"area":null},{"id":"2211.03970","title":"On the Algorithmic Stability and Generalization of Adaptive Optimization Methods","created":"2022-11-08","cited_by_count":5,"area":null},{"id":"2306.07104","title":"Unveiling the Hessian's Connection to the Decision Boundary","created":"2023-06-12","cited_by_count":5,"area":null},{"id":"2308.10103","title":"ASPIRE: Language-Guided Data Augmentation for Improving Robustness Against Spurious Correlations","created":"2023-08-19","cited_by_count":5,"area":null},{"id":"2605.13079","title":"Spectral Flattening Is All Muon Needs: How Orthogonalization Controls Learning Rate and Convergence","created":"2026-05-13","cited_by_count":5,"area":null},{"id":"2106.10704","title":"Better Training using Weight-Constrained Stochastic Dynamics","created":"2021-06-20","cited_by_count":4,"area":null},{"id":"2202.04509","title":"Optimal learning rate schedules in high-dimensional non-convex optimization problems","created":"2022-02-09","cited_by_count":4,"area":null},{"id":"2306.00342","title":"Combining Explicit and Implicit Regularization for Efficient Learning in Deep Networks","created":"2023-06-01","cited_by_count":4,"area":null},{"id":"2402.06697","title":"Feed-Forward Neural Networks as a Mixed-Integer Program","created":"2024-02-09","cited_by_count":4,"area":null}],"next_cursor":"100"}