{"total":78,"rows":[{"id":"2412.13663","title":"Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference","created":"2024-12-18","cited_by_count":554,"area":null},{"id":"2103.00065","title":"Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability","created":"2021-02-26","cited_by_count":427,"area":null},{"id":"2209.15594","title":"Self-Stabilization: The Implicit Bias of Gradient Descent at the Edge of Stability","created":"2022-09-30","cited_by_count":155,"area":null},{"id":"2507.10524","title":"Mixture-of-Recursions: Learning Dynamic Recursive Depths for Adaptive Token-Level Computation","created":"2025-07-14","cited_by_count":133,"area":null},{"id":"2409.16235","title":"EuroLLM: Multilingual Language Models for Europe","created":"2024-09-24","cited_by_count":106,"area":null},{"id":"2302.07011","title":"A Modern Look at the Relationship between Sharpness and Generalization","created":"2023-02-14","cited_by_count":92,"area":null},{"id":"2204.01050","title":"Understanding the unstable convergence of gradient descent","created":"2022-04-03","cited_by_count":86,"area":null},{"id":"2210.05337","title":"SGD with Large Step Sizes Learns Sparse Features","created":"2022-10-11","cited_by_count":68,"area":null},{"id":"2109.14119","title":"Stochastic Training is Not Necessary for Generalization","created":"2021-09-29","cited_by_count":63,"area":null},{"id":"2207.12678","title":"Analyzing Sharpness along GD Trajectory: Progressive Sharpening and Edge of Stability","created":"2022-07-26","cited_by_count":63,"area":null},{"id":"2205.12411","title":"Linear Connectivity Reveals Generalization Strategies","created":"2022-05-24","cited_by_count":56,"area":null},{"id":"2210.03294","title":"Understanding Edge-of-Stability Training Dynamics with a Minimalist Example","created":"2022-10-07","cited_by_count":56,"area":null},{"id":"2212.07469","title":"Learning threshold neurons via the \"edge of stability\"","created":"2022-12-14","cited_by_count":51,"area":null},{"id":"2305.11788","title":"Implicit Bias of Gradient Descent for Logistic Regression at the Edge of Stability","created":"2023-05-19","cited_by_count":42,"area":null},{"id":"2307.06440","title":"No Train No Gain: Revisiting Efficient Training Algorithms For Transformer-based Language Models","created":"2023-07-12","cited_by_count":41,"area":null},{"id":"2102.05375","title":"Strength of Minibatch Noise in SGD","created":"2021-02-10","cited_by_count":39,"area":null},{"id":"2109.06096","title":"The Grammar-Learning Trajectories of Neural Language Models","created":"2021-09-13","cited_by_count":36,"area":null},{"id":"2502.15938","title":"Straight to Zero: Why Linearly Decaying the Learning Rate to Zero Works Best for LLMs","created":"2025-02-21","cited_by_count":35,"area":null},{"id":"2107.11228","title":"Taxonomizing local versus global structure in neural network loss landscapes","created":"2021-07-23","cited_by_count":34,"area":null},{"id":"2306.04815","title":"Catapults in SGD: spikes in the training loss and their impact on generalization through feature learning","created":"2023-06-07","cited_by_count":31,"area":null},{"id":"2108.00781","title":"Generalization Bounds using Lower Tail Exponents in Stochastic Optimizers","created":"2021-08-02","cited_by_count":27,"area":null},{"id":"2307.06966","title":"Layer-wise Linear Mode Connectivity","created":"2023-07-13","cited_by_count":22,"area":null},{"id":"2305.13064","title":"Gradient Descent Monotonically Decreases the Sharpness of Gradient Flow Solutions in Scalar Networks and Beyond","created":"2023-05-22","cited_by_count":21,"area":null},{"id":"2209.03695","title":"Training Scale-Invariant Neural Networks on the Sphere Can Happen in Three Regimes","created":"2022-09-08","cited_by_count":19,"area":null},{"id":"2205.01938","title":"DeepFD: Automated Fault Diagnosis and Localization for Deep Learning Programs","created":"2022-05-04","cited_by_count":16,"area":null},{"id":"2305.12133","title":"Loss Spike in Training Neural Networks","created":"2023-05-20","cited_by_count":16,"area":null},{"id":"2510.04800","title":"Hybrid Architectures for Language Models: Systematic Analysis and Design Insights","created":"2025-10-06","cited_by_count":14,"area":"ssm"},{"id":"2311.04163","title":"Outliers with Opposing Signals Have an Outsized Effect on Neural Network Optimization","created":"2023-11-07","cited_by_count":13,"area":null},{"id":"2506.04805","title":"Adaptive Preconditioners Trigger Loss Spikes in Adam","created":"2025-06-05","cited_by_count":13,"area":null},{"id":"2211.10691","title":"Two Facets of SDE Under an Information-Theoretic Lens: Generalization of SGD via Training Trajectories and via Terminal States","created":"2022-11-19","cited_by_count":11,"area":null},{"id":"2210.12864","title":"K-SAM: Sharpness-Aware Minimization at the Speed of SGD","created":"2022-10-23","cited_by_count":10,"area":null},{"id":"2306.08590","title":"Beyond Implicit Bias: The Insignificance of SGD Noise in Online Learning","created":"2023-06-14","cited_by_count":10,"area":null},{"id":"2403.04085","title":"Don't Blame the Data, Blame the Model: Understanding Noise and Bias When Learning from Subjective Annotations","created":"2024-03-06","cited_by_count":10,"area":null},{"id":"2604.14108","title":"Momentum Further Constrains Sharpness at the Edge of Stochastic Stability","created":"2026-04-15","cited_by_count":10,"area":null},{"id":"2303.05369","title":"Data-dependent Generalization Bounds via Variable-Size Compressibility","created":"2023-03-09","cited_by_count":9,"area":null},{"id":"2507.09846","title":"Through the River: Understanding the Benefit of Schedule-Free Methods for Language Model Training","created":"2025-07-14","cited_by_count":9,"area":null},{"id":"2107.11774","title":"SGD with a Constant Large Learning Rate Can Converge to Local Maxima","created":"2021-07-25","cited_by_count":8,"area":null},{"id":"2202.13733","title":"On the Benefits of Large Learning Rates for Kernel Methods","created":"2022-02-28","cited_by_count":8,"area":null},{"id":"2310.17074","title":"Benign Oscillation of Stochastic Gradient Descent with Large Learning Rates","created":"2023-10-26","cited_by_count":8,"area":null},{"id":"2102.06604","title":"Cockpit: A Practical Debugging Tool for the Training of Deep Neural Networks","created":"2021-02-12","cited_by_count":7,"area":null},{"id":"2504.04105","title":"Minimax Optimal Convergence of Gradient Descent in Logistic Regression via Large and Adaptive Stepsizes","created":"2025-04-05","cited_by_count":7,"area":null},{"id":"2510.00379","title":"Composer: A Search Framework for Hybrid Neural Architecture Design","created":"2025-10-01","cited_by_count":7,"area":null},{"id":"2201.09209","title":"Weight Expansion: A New Perspective on Dropout and Generalization","created":"2022-01-23","cited_by_count":6,"area":null},{"id":"2410.22113","title":"Where Do Large Learning Rates Lead Us?","created":"2024-10-29","cited_by_count":6,"area":null},{"id":"2604.13627","title":"(How) Learning Rates Regulate Catastrophic Overtraining","created":"2026-04-15","cited_by_count":6,"area":null},{"id":"2605.07870","title":"Spectral Dynamics in Deep Networks: Feature Learning, Outlier Escape, and Learning Rate Transfer","created":"2026-05-08","cited_by_count":6,"area":null},{"id":"2601.09000","title":"Universal Dynamics of Warmup Stable Decay: understanding WSD beyond Transformers","created":"2026-01-13","cited_by_count":5,"area":null},{"id":"2503.02809","title":"A Minimalist Example of Edge-of-Stability and Progressive Sharpening","created":"2025-03-04","cited_by_count":4,"area":null},{"id":"2304.04664","title":"Inductive biases in deep learning models for weather prediction","created":"2023-04-06","cited_by_count":3,"area":null},{"id":"2407.01825","title":"Reevaluating Theoretical Analysis Methods for Optimization in Deep Learning","created":"2024-07-01","cited_by_count":3,"area":null},{"id":"2407.02322","title":"Stochastic Differential Equations models for Least-Squares Stochastic Gradient Descent","created":"2024-07-02","cited_by_count":3,"area":null},{"id":"2602.07145","title":"Convex Dominance in Deep Learning I: A Scaling Law of Loss and Learning Rate","created":"2026-02-06","cited_by_count":3,"area":null},{"id":"2604.20614","title":"Too Sharp, Too Sure: When Calibration Follows Curvature","created":"2026-04-22","cited_by_count":3,"area":null},{"id":"2506.18508","title":"Theoretical guarantees for neural estimators in parametric statistics","created":"2025-06-23","cited_by_count":2,"area":null},{"id":"2506.18588","title":"Optimization-Induced Dynamics of Lipschitz Continuity in Neural Networks","created":"2025-06-23","cited_by_count":2,"area":null},{"id":"2602.05600","title":"On the Superlinear Relationship between SGD Noise Covariance and Loss Landscape Curvature","created":"2026-02-05","cited_by_count":2,"area":null},{"id":"2604.19740","title":"Generalization at the Edge of Stability","created":"2026-04-21","cited_by_count":2,"area":null},{"id":"2306.09991","title":"Evolutionary Algorithms in the Light of SGD: Limit Equivalence, Minima Flatness, and Transfer Learning","created":"2023-05-20","cited_by_count":1,"area":null},{"id":"2410.16476","title":"In Search of the Successful Interpolation: On the Role of Sharpness in CLIP Generalization","created":"2024-10-21","cited_by_count":1,"area":null},{"id":"2502.04889","title":"Any-stepsize Gradient Descent for Separable Data under Fenchel-Young Losses","created":"2025-02-07","cited_by_count":1,"area":null},{"id":"2511.04981","title":"Scaling depth capacity via zero/one-layer model expansion","created":"2025-11-07","cited_by_count":1,"area":null},{"id":"2512.23065","title":"TabiBERT: A Large-Scale ModernBERT Foundation Model and A Unified Benchmark for Turkish","created":"2025-12-28","cited_by_count":1,"area":null},{"id":"2603.10079","title":"Large Spikes in Stochastic Gradient Descent: A Large-Deviations View","created":"2026-03-10","cited_by_count":1,"area":null},{"id":"2604.21016","title":"SGD at the Edge of Stability: The Stochastic Sharpness Gap","created":"2026-04-22","cited_by_count":1,"area":null},{"id":"2605.16622","title":"Does Weight Decay Enhance Training Stability?","created":"2026-05-15","cited_by_count":1,"area":null},{"id":"2607.21005","title":"Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay","created":"2026-07-23","cited_by_count":1,"area":null},{"id":"2608.18415","title":"The Road Taken: The Role of Optimizers at the Edge of Stability","created":"2026-08-19","cited_by_count":1,"area":null},{"id":"2609.30957","title":"Towards Understanding Momentum Acceleration in River-Valley Loss Landscape","created":"2026-09-25","cited_by_count":1,"area":null},{"id":"2108.13880","title":"Using a one dimensional parabolic model of the full-batch loss to estimate learning rates during training","created":"2021-08-31","cited_by_count":0,"area":null},{"id":"2410.10690","title":"Dynamical loss functions shape landscape topography and improve learning in artificial neural networks","created":"2024-10-14","cited_by_count":0,"area":null},{"id":"2411.01653","title":"Diagnosing Medical Datasets with Training Dynamics","created":"2024-11-03","cited_by_count":0,"area":null},{"id":"2412.08025","title":"Criteria and Bias of Parameterized Linear Regression under Edge of Stability Regime","created":"2024-12-11","cited_by_count":0,"area":null},{"id":"2606.04212","title":"Edge of Stability Selectively Shapes Learning Across the Data Distribution","created":"2026-06-02","cited_by_count":0,"area":null},{"id":"2608.01284","title":"Training nGPT","created":"2026-08-02","cited_by_count":0,"area":"moe"},{"id":"2609.01034","title":"The Multiple Timescales of Gradient Descent on the Edge of Stability: A Perturbative Derivation of the Central Flow","created":"2026-09-01","cited_by_count":0,"area":null},{"id":"2609.39486","title":"From Wrecks to Wisdom: Recovering Crash Mechanics from Real-World Multi-View Photos","created":"2026-09-30","cited_by_count":0,"area":null},{"id":"2610.06675","title":"Improved Convergence of Large Stepsize Gradient Descent for Logistic Regression","created":"2026-10-05","cited_by_count":0,"area":null},{"id":"2610.06701","title":"A Solvable Model of Adaptive Learning Rate Rescaling: Acceleration, Stability & Scaling","created":"2026-10-05","cited_by_count":0,"area":null}],"next_cursor":null}