{"total":155,"rows":[{"id":"2309.14322","title":"Small-scale proxies for large-scale Transformer training instabilities","created":"2023-09-25","cited_by_count":183,"area":null},{"id":"2302.07011","title":"A Modern Look at the Relationship between Sharpness and Generalization","created":"2023-02-14","cited_by_count":92,"area":null},{"id":"2402.18762","title":"Disentangling the Causes of Plasticity Loss in Neural Networks","created":"2024-02-29","cited_by_count":86,"area":"deeprl"},{"id":"2402.09963","title":"Why are Sensitive Functions Hard for Transformers?","created":"2024-02-15","cited_by_count":74,"area":null},{"id":"2404.04454","title":"Implicit Bias of AdamW: $\\ell_\\infty$ Norm Constrained Optimization","created":"2024-04-05","cited_by_count":58,"area":null},{"id":"2210.01513","title":"The Dynamics of Sharpness-Aware Minimization: Bouncing Across Ravines and Drifting Towards Wide Minima","created":"2022-10-04","cited_by_count":51,"area":null},{"id":"2212.07469","title":"Learning threshold neurons via the \"edge of stability\"","created":"2022-12-14","cited_by_count":51,"area":null},{"id":"2406.09405","title":"Why Warmup the Learning Rate? Underlying Mechanisms and Improvements","created":"2024-06-13","cited_by_count":46,"area":null},{"id":"2304.03408","title":"Dynamics of Finite Width Kernel and Prediction Fluctuations in Mean Field Neural Networks","created":"2023-04-06","cited_by_count":43,"area":null},{"id":"2305.11788","title":"Implicit Bias of Gradient Descent for Logistic Regression at the Edge of Stability","created":"2023-05-19","cited_by_count":42,"area":null},{"id":"2210.04860","title":"Second-order regression models exhibit progressive sharpening to the edge of stability","created":"2022-10-10","cited_by_count":40,"area":null},{"id":"2302.08982","title":"(S)GD over Diagonal Linear Networks: Implicit Regularisation, Large Stepsizes and Edge of Stability","created":"2023-02-17","cited_by_count":36,"area":null},{"id":"2405.16002","title":"Does SGD really happen in tiny subspaces?","created":"2024-05-25","cited_by_count":36,"area":null},{"id":"2305.16284","title":"DoWG Unleashed: An Efficient Universal Parameter-Free Gradient Descent Method","created":"2023-05-25","cited_by_count":35,"area":null},{"id":"2402.15926","title":"Large Stepsize Gradient Descent for Logistic Loss: Non-Monotonicity of the Loss Improves Optimization Efficiency","created":"2024-02-24","cited_by_count":35,"area":null},{"id":"2306.04815","title":"Catapults in SGD: spikes in the training loss and their impact on generalization through feature learning","created":"2023-06-07","cited_by_count":31,"area":null},{"id":"2604.21691","title":"There Will Be a Scientific Theory of Deep Learning","created":"2026-04-23","cited_by_count":29,"area":null},{"id":"2305.15287","title":"The Crucial Role of Normalization in Sharpness-Aware Minimization","created":"2023-05-24","cited_by_count":26,"area":null},{"id":"2307.04204","title":"Trajectory Alignment: Understanding the Edge of Stability Phenomenon via Bifurcation Theory","created":"2023-07-09","cited_by_count":26,"area":null},{"id":"2310.17087","title":"Good regularity creates large learning rate implicit biases: edge of stability, balancing, and catapult","created":"2023-10-26","cited_by_count":25,"area":null},{"id":"2302.08692","title":"SAM operates far from home: eigenvalue regularization as a dynamical phenomenon","created":"2023-02-17","cited_by_count":24,"area":null},{"id":"2402.17457","title":"Super Consistency of Neural Network Landscapes and Learning Rate Transfer","created":"2024-02-27","cited_by_count":23,"area":null},{"id":"2409.15156","title":"Rethinking Conventional Wisdom in Machine Learning: From Generalization to Scaling","created":"2024-09-23","cited_by_count":23,"area":"lr"},{"id":"2311.02076","title":"Universal Sharpness Dynamics in Neural Network Training: Fixed Point Analysis, Edge of Stability, and Route to Chaos","created":"2023-11-03","cited_by_count":22,"area":null},{"id":"2405.14578","title":"Surge Phenomenon in Optimal Learning Rate and Batch Size Scaling","created":"2024-05-23","cited_by_count":22,"area":null},{"id":"2406.06893","title":"Transformers Provably Learn Sparse Token Selection While Fully-Connected Nets Cannot","created":"2024-06-11","cited_by_count":22,"area":null},{"id":"2503.12811","title":"A Multi-Power Law for Loss Curve Prediction Across Learning Rate Schedules","created":"2025-03-17","cited_by_count":22,"area":null},{"id":"2301.02312","title":"Training trajectories, mini-batch losses and the curious role of the learning rate","created":"2023-01-05","cited_by_count":21,"area":null},{"id":"2305.13064","title":"Gradient Descent Monotonically Decreases the Sharpness of Gradient Flow Solutions in Scalar Networks and Beyond","created":"2023-05-22","cited_by_count":21,"area":null},{"id":"2405.03676","title":"Why is SAM Robust to Label Noise?","created":"2024-05-06","cited_by_count":21,"area":null},{"id":"2501.12666","title":"Explicit Eigenvalue Regularization Improves Sharpness-Aware Minimization","created":"2025-01-22","cited_by_count":21,"area":null},{"id":"2302.12250","title":"Phase diagram of early training dynamics in deep neural networks: effect of the learning rate, depth, and width","created":"2023-02-23","cited_by_count":19,"area":null},{"id":"2404.19261","title":"High dimensional analysis reveals conservative sharpening and a stochastic edge of stability","created":"2024-04-30","cited_by_count":18,"area":null},{"id":"2309.12488","title":"Sharpness-Aware Minimization and the Edge of Stability","created":"2023-09-21","cited_by_count":17,"area":null},{"id":"2310.01687","title":"From Stability to Chaos: Analyzing Gradient Descent Dynamics in Quadratic Regression","created":"2023-10-02","cited_by_count":17,"area":null},{"id":"2405.13456","title":"Deep linear networks for regression are implicitly regularized towards flat minima","created":"2024-05-22","cited_by_count":17,"area":null},{"id":"2406.08654","title":"Large Stepsize Gradient Descent for Non-Homogeneous Two-Layer Networks: Margin Improvement and Fast Optimization","created":"2024-06-12","cited_by_count":17,"area":null},{"id":"2305.12133","title":"Loss Spike in Training Neural Networks","created":"2023-05-20","cited_by_count":16,"area":null},{"id":"2410.04887","title":"Wide Neural Networks Trained with Weight Decay Provably Exhibit Neural Collapse","created":"2024-10-07","cited_by_count":16,"area":null},{"id":"2305.12467","title":"Understanding Multi-phase Optimization Dynamics and Rich Nonlinear Behaviors of ReLU Networks","created":"2023-05-21","cited_by_count":15,"area":null},{"id":"2406.05033","title":"Gradient Descent on Logistic Regression with Non-Separable Data and Large Step Sizes","created":"2024-06-07","cited_by_count":14,"area":null},{"id":"2607.21716","title":"A Defense of the Quadratic Model","created":"2026-07-23","cited_by_count":14,"area":null},{"id":"2305.15659","title":"How to escape sharp minima with random perturbations","created":"2023-05-25","cited_by_count":13,"area":null},{"id":"2311.04163","title":"Outliers with Opposing Signals Have an Outsized Effect on Neural Network Optimization","created":"2023-11-07","cited_by_count":13,"area":null},{"id":"2403.18494","title":"Learning in PINNs: Phase transition, diffusion equilibrium, and generalization","created":"2024-03-27","cited_by_count":13,"area":null},{"id":"2410.04642","title":"The Optimization Landscape of SGD Across the Feature Learning Strength","created":"2024-10-06","cited_by_count":13,"area":null},{"id":"2504.13292","title":"Let Me Grok for You: Accelerating Grokking via Embedding Transfer from a Weaker Model","created":"2025-04-17","cited_by_count":13,"area":null},{"id":"2506.04805","title":"Adaptive Preconditioners Trigger Loss Spikes in Adam","created":"2025-06-05","cited_by_count":13,"area":null},{"id":"2406.06838","title":"Stable Minima Cannot Overfit in Univariate ReLU Networks: Generalization by Large Step Sizes","created":"2024-06-10","cited_by_count":12,"area":null},{"id":"2302.01952","title":"On a continuous time model of gradient descent dynamics and instability in deep learning","created":"2023-02-03","cited_by_count":11,"area":null},{"id":"2306.07850","title":"Exact Mean Square Linear Stability Analysis for SGD","created":"2023-06-13","cited_by_count":11,"area":null},{"id":"2602.03702","title":"Anytime Pretraining: Horizon-Free Learning-Rate Schedules with Weight Averaging","created":"2026-02-03","cited_by_count":10,"area":null},{"id":"2604.14108","title":"Momentum Further Constrains Sharpness at the Edge of Stochastic Stability","created":"2026-04-15","cited_by_count":10,"area":null},{"id":"2605.02105","title":"Sharpness-Aware Pretraining Mitigates Catastrophic Forgetting","created":"2026-05-04","cited_by_count":10,"area":null},{"id":"2410.14067","title":"Provable Benefits of Complex Parameterizations for Structured State Space Models","created":"2024-10-17","cited_by_count":9,"area":"ssm"},{"id":"2505.12387","title":"Neural Thermodynamics: Entropic Forces in Deep and Universal Representation Learning","created":"2025-05-18","cited_by_count":9,"area":null},{"id":"2506.02336","title":"Large Stepsizes Accelerate Gradient Descent for Regularized Logistic Regression","created":"2025-06-03","cited_by_count":9,"area":null},{"id":"2507.09846","title":"Through the River: Understanding the Benefit of Schedule-Free Methods for Language Model Training","created":"2025-07-14","cited_by_count":9,"area":null},{"id":"2602.01480","title":"Rod Flow: A Continuous-Time Model for Gradient Descent at the Edge of Stability","created":"2026-02-01","cited_by_count":9,"area":null},{"id":"2603.05002","title":"Non-Euclidean Gradient Descent Operates at the Edge of Stability","created":"2026-03-05","cited_by_count":9,"area":null},{"id":"2406.11110","title":"How Neural Networks Learn the Support is an Implicit Regularization Effect of SGD","created":"2024-06-17","cited_by_count":8,"area":null},{"id":"2407.06183","title":"Stepping on the Edge: Curvature Aware Learning Rate Tuners","created":"2024-07-08","cited_by_count":8,"area":null},{"id":"2505.09022","title":"Block-Biased Mamba for Long-Range Sequence Processing","created":"2025-05-13","cited_by_count":8,"area":"ssm"},{"id":"2601.16979","title":"A Scalable Measure of Loss Landscape Curvature for Analyzing the Training Dynamics of LLMs","created":"2026-01-23","cited_by_count":8,"area":null},{"id":"2210.05371","title":"On skip connections and normalisation layers in deep optimisation","created":"2022-10-10","cited_by_count":7,"area":null},{"id":"2305.14683","title":"On progressive sharpening, flat minima and generalisation","created":"2023-05-24","cited_by_count":7,"area":null},{"id":"2504.04105","title":"Minimax Optimal Convergence of Gradient Descent in Logistic Regression via Large and Adaptive Stepsizes","created":"2025-04-05","cited_by_count":7,"area":null},{"id":"2509.25351","title":"Gradient Descent with Large Step Sizes: Chaos and Fractal Convergence Region","created":"2025-09-29","cited_by_count":7,"area":null},{"id":"2512.22768","title":"Understanding the Mechanisms of Fast Hyperparameter Transfer","created":"2025-12-28","cited_by_count":7,"area":null},{"id":"2306.13239","title":"The Inductive Bias of Flatness Regularization for Deep Matrix Factorization","created":"2023-06-22","cited_by_count":6,"area":null},{"id":"2401.11600","title":"Understanding the Generalization Benefits of Late Learning Rate Decay","created":"2024-01-21","cited_by_count":6,"area":null},{"id":"2502.02132","title":"How Memory in Optimization Algorithms Implicitly Modifies the Loss","created":"2025-02-04","cited_by_count":6,"area":null},{"id":"2506.05454","title":"Zeroth-Order Optimization Finds Flat Minima","created":"2025-06-05","cited_by_count":6,"area":"deeprl"},{"id":"2506.16884","title":"The Importance of Being Lazy: Scaling Limits of Continual Learning","created":"2025-06-20","cited_by_count":6,"area":null},{"id":"2507.12837","title":"Understanding the Evolution of the Neural Tangent Kernel at the Edge of Stability","created":"2025-07-17","cited_by_count":6,"area":null},{"id":"2604.20446","title":"The Origin of Edge of Stability","created":"2026-04-22","cited_by_count":6,"area":null},{"id":"2310.14423","title":"A Quadratic Synchronization Rule for Distributed Deep Learning","created":"2023-10-22","cited_by_count":5,"area":null},{"id":"2506.06940","title":"Understanding Sharpness Dynamics in NN Training with a Minimalist Example: The Effects of Dataset Difficulty, Depth, Stochasticity, and More","created":"2025-06-07","cited_by_count":5,"area":null},{"id":"2602.05813","title":"Where Does Warm-Up Come From? Adaptive Scheduling for Norm-Constrained Optimizers","created":"2026-02-05","cited_by_count":5,"area":null},{"id":"2312.00209","title":"On the Interplay Between Stepsize Tuning and Progressive Sharpening","created":"2023-11-30","cited_by_count":4,"area":null},{"id":"2410.16401","title":"Simplicity Bias via Global Convergence of Sharpness Minimization","created":"2024-10-21","cited_by_count":4,"area":null},{"id":"2503.02809","title":"A Minimalist Example of Edge-of-Stability and Progressive Sharpening","created":"2025-03-04","cited_by_count":4,"area":null},{"id":"2505.21722","title":"Saddle-To-Saddle Dynamics in Deep ReLU Networks: Low-Rank Bias in the First Saddle Escape","created":"2025-05-27","cited_by_count":4,"area":null},{"id":"2506.20752","title":"Characterization and Mitigation of Training Instabilities in Microscaling Formats","created":"2025-06-25","cited_by_count":4,"area":null},{"id":"2510.17506","title":"Convergence Rates for Gradient Descent on the Edge of Stability in Overparametrised Least Squares","created":"2025-10-20","cited_by_count":4,"area":null},{"id":"2604.09258","title":"Nexus: Same Pretraining Loss, Better Downstream Generalization via Common Minima","created":"2026-04-10","cited_by_count":4,"area":null},{"id":"2605.22432","title":"AMUSE: Anytime Muon with Stable Gradient Evaluation","created":"2026-05-21","cited_by_count":4,"area":null},{"id":"2311.15051","title":"Gradient Descent with Polyak's Momentum Finds Flatter Minima via Large Catapults","created":"2023-11-25","cited_by_count":3,"area":null},{"id":"2410.10473","title":"The Implicit Bias of Structured State Space Models Can Be Poisoned With Clean Labels","created":"2024-10-14","cited_by_count":3,"area":"ssm"},{"id":"2505.13900","title":"New Evidence of the Two-Phase Learning Dynamics of Neural Networks","created":"2025-05-20","cited_by_count":3,"area":null},{"id":"2510.18120","title":"Generalization Below the Edge of Stability: The Role of Data Geometry","created":"2025-10-20","cited_by_count":3,"area":null},{"id":"2510.25244","title":"BSFA: Leveraging the Subspace Dichotomy to Accelerate Neural Network Training","created":"2025-10-29","cited_by_count":3,"area":null},{"id":"2602.14789","title":"On the Stability of Nonlinear Dynamics in GD and SGD: Beyond Quadratic Potentials","created":"2026-02-16","cited_by_count":3,"area":null},{"id":"2606.18080","title":"Edge Flow: A Tractable and Predictive Continuous-Time Model for Gradient Descent at the Edge of Stability","created":"2026-06-16","cited_by_count":3,"area":null},{"id":"2505.11370","title":"Understanding Nonlinear Implicit Bias via Region Counts in Input Space","created":"2025-05-16","cited_by_count":2,"area":null},{"id":"2506.12903","title":"Variational Learning Finds Flatter Solutions at the Edge of Stability","created":"2025-06-15","cited_by_count":2,"area":null},{"id":"2506.13974","title":"Constant Stepsize Local GD for Logistic Regression: Acceleration by Instability","created":"2025-06-16","cited_by_count":2,"area":null},{"id":"2506.18588","title":"Optimization-Induced Dynamics of Lipschitz Continuity in Neural Networks","created":"2025-06-23","cited_by_count":2,"area":null},{"id":"2507.03545","title":"DOME: Improving Signal-to-Noise in Stochastic Gradient Descent via Sharp-Direction Subspace Filtering","created":"2025-07-04","cited_by_count":2,"area":null},{"id":"2509.18001","title":"Unveiling m-Sharpness Through the Structure of Stochastic Gradient Noise","created":"2025-09-22","cited_by_count":2,"area":null}],"next_cursor":"100"}