{"total":101,"rows":[{"id":"2306.14048","title":"H$_2$O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models","created":"2023-06-24","cited_by_count":832,"area":"inference"},{"id":"2309.14322","title":"Small-scale proxies for large-scale Transformer training instabilities","created":"2023-09-25","cited_by_count":183,"area":null},{"id":"2303.06296","title":"Stabilizing Transformer Training by Preventing Attention Entropy Collapse","created":"2023-03-11","cited_by_count":161,"area":null},{"id":"2209.15594","title":"Self-Stabilization: The Implicit Bias of Gradient Descent at the Edge of Stability","created":"2022-09-30","cited_by_count":155,"area":null},{"id":"2304.13013","title":"Stable and low-precision training for large-scale vision-language models","created":"2023-04-25","cited_by_count":75,"area":"mllm"},{"id":"2310.04415","title":"Why Do We Need Weight Decay in Modern Deep Learning?","created":"2023-10-06","cited_by_count":72,"area":null},{"id":"2303.01610","title":"Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformers","created":"2023-03-02","cited_by_count":59,"area":"moe"},{"id":"2306.00204","title":"Toward Understanding Why Adam Converges Faster Than SGD for Transformers","created":"2023-05-31","cited_by_count":53,"area":null},{"id":"2501.16371","title":"Optimizing the Optimizer for Physics-Informed Neural Networks and Kolmogorov-Arnold Networks","created":"2025-01-22","cited_by_count":42,"area":null},{"id":"2210.04860","title":"Second-order regression models exhibit progressive sharpening to the edge of stability","created":"2022-10-10","cited_by_count":40,"area":null},{"id":"2305.16284","title":"DoWG Unleashed: An Efficient Universal Parameter-Free Gradient Descent Method","created":"2023-05-25","cited_by_count":35,"area":null},{"id":"2502.19002","title":"The Sharpness Disparity Principle in Transformers for Accelerating Language Model Pre-Training","created":"2025-02-26","cited_by_count":30,"area":null},{"id":"2309.00079","title":"On the Implicit Bias of Adam","created":"2023-08-31","cited_by_count":27,"area":null},{"id":"2302.08692","title":"SAM operates far from home: eigenvalue regularization as a dynamical phenomenon","created":"2023-02-17","cited_by_count":24,"area":null},{"id":"2402.17457","title":"Super Consistency of Neural Network Landscapes and Learning Rate Transfer","created":"2024-02-27","cited_by_count":23,"area":null},{"id":"2409.15156","title":"Rethinking Conventional Wisdom in Machine Learning: From Generalization to Scaling","created":"2024-09-23","cited_by_count":23,"area":"lr"},{"id":"2405.14578","title":"Surge Phenomenon in Optimal Learning Rate and Batch Size Scaling","created":"2024-05-23","cited_by_count":22,"area":null},{"id":"2507.02119","title":"Scaling Collapse Reveals Universal Dynamics in Compute-Optimally Trained Neural Networks","created":"2025-07-02","cited_by_count":22,"area":null},{"id":"2305.13064","title":"Gradient Descent Monotonically Decreases the Sharpness of Gradient Flow Solutions in Scalar Networks and Beyond","created":"2023-05-22","cited_by_count":21,"area":null},{"id":"2306.09000","title":"When and Why Momentum Accelerates SGD:An Empirical Study","created":"2023-06-15","cited_by_count":19,"area":null},{"id":"2404.19261","title":"High dimensional analysis reveals conservative sharpening and a stochastic edge of stability","created":"2024-04-30","cited_by_count":18,"area":null},{"id":"2405.12807","title":"FAdam: Adam is a natural gradient optimizer using diagonal empirical Fisher information","created":"2024-05-21","cited_by_count":18,"area":null},{"id":"2309.12488","title":"Sharpness-Aware Minimization and the Edge of Stability","created":"2023-09-21","cited_by_count":17,"area":null},{"id":"2403.04081","title":"Directional Smoothness and Gradient Methods: Convergence and Adaptivity","created":"2024-03-06","cited_by_count":17,"area":null},{"id":"2302.07384","title":"The Geometry of Neural Nets' Parameter Spaces Under Reparametrization","created":"2023-02-14","cited_by_count":16,"area":null},{"id":"2510.25000","title":"What Really Matters in Matrix-Whitening Optimizers?","created":"2025-10-28","cited_by_count":16,"area":null},{"id":"2607.21716","title":"A Defense of the Quadratic Model","created":"2026-07-23","cited_by_count":14,"area":null},{"id":"2311.04163","title":"Outliers with Opposing Signals Have an Outsized Effect on Neural Network Optimization","created":"2023-11-07","cited_by_count":13,"area":null},{"id":"2403.18494","title":"Learning in PINNs: Phase transition, diffusion equilibrium, and generalization","created":"2024-03-27","cited_by_count":13,"area":null},{"id":"2505.02809","title":"Towards Quantifying the Hessian Structure of Neural Networks","created":"2025-05-05","cited_by_count":13,"area":null},{"id":"2506.04805","title":"Adaptive Preconditioners Trigger Loss Spikes in Adam","created":"2025-06-05","cited_by_count":13,"area":null},{"id":"2212.07295","title":"Maximal Initial Learning Rates in Deep ReLU Networks","created":"2022-12-14","cited_by_count":11,"area":null},{"id":"2302.09693","title":"mSAM: Micro-Batch-Averaged Sharpness-Aware Minimization","created":"2023-02-19","cited_by_count":10,"area":null},{"id":"2504.07379","title":"Representation Meets Optimization: Training PINNs and PIKANs for Gray-Box Discovery in Systems Pharmacology","created":"2025-04-10","cited_by_count":10,"area":null},{"id":"2604.14108","title":"Momentum Further Constrains Sharpness at the Edge of Stochastic Stability","created":"2026-04-15","cited_by_count":10,"area":null},{"id":"2507.09846","title":"Through the River: Understanding the Benefit of Schedule-Free Methods for Language Model Training","created":"2025-07-14","cited_by_count":9,"area":null},{"id":"2603.05002","title":"Non-Euclidean Gradient Descent Operates at the Edge of Stability","created":"2026-03-05","cited_by_count":9,"area":null},{"id":"2310.17074","title":"Benign Oscillation of Stochastic Gradient Descent with Large Learning Rates","created":"2023-10-26","cited_by_count":8,"area":null},{"id":"2407.06183","title":"Stepping on the Edge: Curvature Aware Learning Rate Tuners","created":"2024-07-08","cited_by_count":8,"area":null},{"id":"2412.11768","title":"No More Adam: Learning Rate Scaling at Initialization is All You Need","created":"2024-12-16","cited_by_count":8,"area":null},{"id":"2505.24275","title":"GradPower: Powering Gradients for Faster Language Model Pre-Training","created":"2025-05-30","cited_by_count":8,"area":null},{"id":"2601.16979","title":"A Scalable Measure of Loss Landscape Curvature for Analyzing the Training Dynamics of LLMs","created":"2026-01-23","cited_by_count":8,"area":null},{"id":"2411.12135","title":"Exact Risk Curves of signSGD in High-Dimensions: Quantifying Preconditioning and Noise-Compression Effects","created":"2024-11-19","cited_by_count":7,"area":null},{"id":"2306.13239","title":"The Inductive Bias of Flatness Regularization for Deep Matrix Factorization","created":"2023-06-22","cited_by_count":6,"area":null},{"id":"2409.07787","title":"Stable Language Model Pre-training by Reducing Embedding Variability","created":"2024-09-12","cited_by_count":6,"area":null},{"id":"2410.22568","title":"Fast Deep Hedging with Second-Order Optimization","created":"2024-10-29","cited_by_count":6,"area":null},{"id":"2502.02132","title":"How Memory in Optimization Algorithms Implicitly Modifies the Loss","created":"2025-02-04","cited_by_count":6,"area":null},{"id":"2604.13627","title":"(How) Learning Rates Regulate Catastrophic Overtraining","created":"2026-04-15","cited_by_count":6,"area":null},{"id":"2605.07870","title":"Spectral Dynamics in Deep Networks: Feature Learning, Outlier Escape, and Learning Rate Transfer","created":"2026-05-08","cited_by_count":6,"area":null},{"id":"2306.00342","title":"Combining Explicit and Implicit Regularization for Efficient Learning in Deep Networks","created":"2023-06-01","cited_by_count":4,"area":null},{"id":"2312.00209","title":"On the Interplay Between Stepsize Tuning and Progressive Sharpening","created":"2023-11-30","cited_by_count":4,"area":null},{"id":"2410.16401","title":"Simplicity Bias via Global Convergence of Sharpness Minimization","created":"2024-10-21","cited_by_count":4,"area":null},{"id":"2603.09581","title":"Towards Understanding Adam Convergence on Highly Degenerate Polynomials","created":"2026-03-10","cited_by_count":4,"area":null},{"id":"2605.06152","title":"Grokking or Glitching? How Low-Precision Drives Slingshot Loss Spikes","created":"2026-05-07","cited_by_count":4,"area":null},{"id":"2311.15051","title":"Gradient Descent with Polyak's Momentum Finds Flatter Minima via Large Catapults","created":"2023-11-25","cited_by_count":3,"area":null},{"id":"2402.13108","title":"On the Convergence of Gradient Descent for Large Learning Rates","created":"2024-02-20","cited_by_count":3,"area":null},{"id":"2407.01825","title":"Reevaluating Theoretical Analysis Methods for Optimization in Deep Learning","created":"2024-07-01","cited_by_count":3,"area":null},{"id":"2503.01843","title":"When Can You Get Away with Low Memory Adam?","created":"2025-03-03","cited_by_count":3,"area":null},{"id":"2604.20614","title":"Too Sharp, Too Sure: When Calibration Follows Curvature","created":"2026-04-22","cited_by_count":3,"area":null},{"id":"2605.06821","title":"A Rod Flow Model for Adam at the Edge of Stability","created":"2026-05-07","cited_by_count":3,"area":null},{"id":"2605.09552","title":"Phases of Muon: When Muon Eclipses SignSGD","created":"2026-05-10","cited_by_count":3,"area":null},{"id":"2606.09658","title":"Muon Learns More Robust and Transferable Features than Adam","created":"2026-06-08","cited_by_count":3,"area":null},{"id":"2401.12332","title":"A Precise Characterization of SGD Stability Using Loss Surface Geometry","created":"2024-01-22","cited_by_count":2,"area":null},{"id":"2506.12903","title":"Variational Learning Finds Flatter Solutions at the Edge of Stability","created":"2025-06-15","cited_by_count":2,"area":null},{"id":"2606.19179","title":"Compute Efficiency and Serial Runtime Tradeoffs for Stochastic Momentum Methods","created":"2026-06-17","cited_by_count":2,"area":null},{"id":"2607.00510","title":"Prototype Language Models","created":"2026-07-01","cited_by_count":2,"area":null},{"id":"2608.15448","title":"Language models suffer from a curse of ambiguity","created":"2026-08-15","cited_by_count":2,"area":null},{"id":"2309.05077","title":"Generalization error bounds for iterative learning algorithms with bounded updates","created":"2023-09-10","cited_by_count":1,"area":null},{"id":"2406.00127","title":"Training on the Edge of Stability Is Caused by Layerwise Jacobian Alignment","created":"2024-05-31","cited_by_count":1,"area":null},{"id":"2505.13196","title":"A Physics-Inspired Optimizer: Velocity Regularized Adam","created":"2025-05-19","cited_by_count":1,"area":null},{"id":"2512.19019","title":"Optimizer Dynamics at the Edge of Stability with Differential Privacy","created":"2025-12-22","cited_by_count":1,"area":null},{"id":"2602.05065","title":"Does SGD Seek Flatness or Sharpness? An Exactly Solvable Model","created":"2026-02-04","cited_by_count":1,"area":null},{"id":"2602.12471","title":"Tight Bounds for Logistic Regression with Large Stepsize Gradient Descent in Low Dimension","created":"2026-02-12","cited_by_count":1,"area":null},{"id":"2603.26954","title":"High dimensional theory of two-phase optimizers","created":"2026-03-27","cited_by_count":1,"area":null},{"id":"2605.16622","title":"Does Weight Decay Enhance Training Stability?","created":"2026-05-15","cited_by_count":1,"area":null},{"id":"2608.18415","title":"The Road Taken: The Role of Optimizers at the Edge of Stability","created":"2026-08-19","cited_by_count":1,"area":null},{"id":"2608.20638","title":"Adam at the Edge of Stability: Adaptive Feedback, Provable Oscillation, and Gradient Reversal","created":"2026-08-21","cited_by_count":1,"area":null},{"id":"2608.22129","title":"Blockwise Stabilized Adaptive Cubic Regularization with Subsolvers via Recurrence","created":"2026-08-22","cited_by_count":1,"area":null},{"id":"2307.04210","title":"Investigating the Edge of Stability Phenomenon in Reinforcement Learning","created":"2023-07-09","cited_by_count":0,"area":null},{"id":"2307.09638","title":"Promoting Exploration in Memory-Augmented Adam using Critical Momenta","created":"2023-07-18","cited_by_count":0,"area":null},{"id":"2412.08025","title":"Criteria and Bias of Parameterized Linear Regression under Edge of Stability Regime","created":"2024-12-11","cited_by_count":0,"area":null},{"id":"2510.09034","title":"Convergence of optimizers implies eigenvalues filtering at equilibrium","created":"2025-10-10","cited_by_count":0,"area":null},{"id":"2510.24466","title":"Non-Singularity of the Gradient Descent map for Neural Networks with Piecewise Analytic Activations","created":"2025-10-28","cited_by_count":0,"area":null},{"id":"2603.02620","title":"Same Error, Different Function: The Optimizer as an Implicit Prior in Financial Time Series","created":"2026-03-03","cited_by_count":0,"area":null},{"id":"2603.10301","title":"What do near-optimal learning rate schedules look like?","created":"2026-03-11","cited_by_count":0,"area":null},{"id":"2604.14669","title":"Zeroth-Order Optimization at the Edge of Stability","created":"2026-04-16","cited_by_count":0,"area":null},{"id":"2606.04212","title":"Edge of Stability Selectively Shapes Learning Across the Data Distribution","created":"2026-06-02","cited_by_count":0,"area":null},{"id":"2606.16768","title":"Taming Curvature: Architecture Warm-Up for Stable Transformer Training","created":"2026-06-15","cited_by_count":0,"area":null},{"id":"2608.21660","title":"Variational Structure at the Edge of Stability","created":"2026-08-21","cited_by_count":0,"area":null},{"id":"2608.25873","title":"How Edge of Stability Hinders SCAFFOLD in Federated Optimization","created":"2026-08-26","cited_by_count":0,"area":null},{"id":"2609.18314","title":"Beyond Quadratic Loss: The Stability Phase Diagram of Adam","created":"2026-09-16","cited_by_count":0,"area":null},{"id":"2609.32290","title":"A Journey to the Edge of Stability","created":"2026-09-26","cited_by_count":0,"area":null},{"id":"2609.34130","title":"Training and Inference Dynamics of PLDR-LLMs: Row-Map Collapse, Renormalization, and Predictive Reduction","created":"2026-09-28","cited_by_count":0,"area":null},{"id":"2609.34915","title":"Muon Sublates the Edge of Stability in LLM Pretraining","created":"2026-09-28","cited_by_count":0,"area":null},{"id":"2609.39767","title":"How Does Local Landscape Geometry Evolve in Language Model Pre-Training?","created":"2026-09-30","cited_by_count":0,"area":null},{"id":"2610.01459","title":"Tight Transition Time Bounds for Separable Logistic Regression at the Edge of Stability","created":"2026-10-01","cited_by_count":0,"area":null},{"id":"2610.06120","title":"Mind the Drift: Diagonal Linear Networks Under Large Learning Rates","created":"2026-10-05","cited_by_count":0,"area":null},{"id":"2610.06675","title":"Improved Convergence of Large Stepsize Gradient Descent for Logistic Regression","created":"2026-10-05","cited_by_count":0,"area":null},{"id":"2610.06701","title":"A Solvable Model of Adaptive Learning Rate Rescaling: Acceleration, Stability & Scaling","created":"2026-10-05","cited_by_count":0,"area":null},{"id":"2610.13524","title":"Simulating Training Spikes of Deep Neural Networks by Layer-wise Random Feature Models","created":"2026-10-09","cited_by_count":0,"area":null}],"next_cursor":"100"}