{"total":62,"rows":[{"id":"2205.09745","title":"Understanding Gradient Descent on Edge of Stability in Deep Learning","created":"2022-05-19","cited_by_count":128,"area":null},{"id":"2210.09221","title":"Vision Transformers provably learn spatial structure","created":"2022-10-13","cited_by_count":93,"area":null},{"id":"2206.07085","title":"Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction","created":"2022-06-14","cited_by_count":86,"area":null},{"id":"2102.12470","title":"On the Validity of Modeling SGD with Stochastic Differential Equations (SDEs)","created":"2021-02-24","cited_by_count":73,"area":null},{"id":"2503.10622","title":"Transformers without Normalization","created":"2025-03-13","cited_by_count":70,"area":null},{"id":"2201.05125","title":"GradMax: Growing Neural Networks using Gradient Information","created":"2022-01-13","cited_by_count":66,"area":null},{"id":"2105.05246","title":"Spectral Normalisation for Deep Reinforcement Learning: an Optimisation Perspective","created":"2021-05-11","cited_by_count":65,"area":null},{"id":"2404.04454","title":"Implicit Bias of AdamW: $\\ell_\\infty$ Norm Constrained Optimization","created":"2024-04-05","cited_by_count":58,"area":null},{"id":"2210.03294","title":"Understanding Edge-of-Stability Training Dynamics with a Minimalist Example","created":"2022-10-07","cited_by_count":56,"area":null},{"id":"2105.02716","title":"Noether's Learning Dynamics: Role of Symmetry Breaking in Neural Networks","created":"2021-05-06","cited_by_count":50,"area":null},{"id":"2102.08098","title":"GradInit: Learning to Initialize Neural Networks for Stable and Efficient Training","created":"2021-02-16","cited_by_count":44,"area":null},{"id":"2303.15725","title":"Solving Regularized Exp, Cosh and Sinh Regression Problems","created":"2023-03-28","cited_by_count":44,"area":null},{"id":"2405.13698","title":"How to set AdamW's weight decay as you scale model and dataset size","created":"2024-05-22","cited_by_count":43,"area":null},{"id":"2207.05931","title":"Towards understanding how momentum improves generalization in deep learning","created":"2022-07-13","cited_by_count":40,"area":null},{"id":"2305.16284","title":"DoWG Unleashed: An Efficient Universal Parameter-Free Gradient Descent Method","created":"2023-05-25","cited_by_count":35,"area":null},{"id":"2406.02997","title":"Residual Connections and Normalization Can Provably Prevent Oversmoothing in GNNs","created":"2024-06-05","cited_by_count":34,"area":null},{"id":"2106.03970","title":"Batch Normalization Orthogonalizes Representations in Deep Random Networks","created":"2021-06-07","cited_by_count":29,"area":null},{"id":"2202.00980","title":"Robust Training of Neural Networks Using Scale Invariant Architectures","created":"2022-02-02","cited_by_count":29,"area":null},{"id":"2306.04226","title":"Normalization Layers Are All That Sharpness-Aware Minimization Needs","created":"2023-06-07","cited_by_count":27,"area":null},{"id":"2106.05956","title":"Beyond BatchNorm: Towards a Unified Understanding of Normalization in Deep Learning","created":"2021-06-10","cited_by_count":25,"area":null},{"id":"2103.01499","title":"Demystifying Batch Normalization in ReLU Networks: Equivalent Convex Optimization Models and Implicit Regularization","created":"2021-03-02","cited_by_count":21,"area":null},{"id":"2106.15739","title":"On the Periodic Behavior of Neural Network Training with Batch Normalization and Weight Decay","created":"2021-06-29","cited_by_count":20,"area":null},{"id":"2510.19093","title":"Weight Decay may matter more than muP for Learning Rate Transfer in Practice","created":"2025-10-21","cited_by_count":20,"area":null},{"id":"2209.03695","title":"Training Scale-Invariant Neural Networks on the Sphere Can Happen in Three Regimes","created":"2022-09-08","cited_by_count":19,"area":null},{"id":"2210.03277","title":"Rethinking Normalization Methods in Federated Learning","created":"2022-10-07","cited_by_count":19,"area":null},{"id":"2303.11135","title":"TWINS: A Fine-Tuning Framework for Improved Transferability of Adversarial Robustness and Generalization","created":"2023-03-20","cited_by_count":18,"area":null},{"id":"2302.07384","title":"The Geometry of Neural Nets' Parameter Spaces Under Reparametrization","created":"2023-02-14","cited_by_count":16,"area":null},{"id":"2308.09543","title":"Latent State Models of Training Dynamics","created":"2023-08-18","cited_by_count":16,"area":null},{"id":"2210.15898","title":"Toward Equation of Motion for Deep Neural Networks: Continuous-time Gradient Descent and Discretization Error Analysis","created":"2022-10-28","cited_by_count":13,"area":null},{"id":"2410.17174","title":"From Attention to Activation: Unravelling the Enigmas of Large Language Models","created":"2024-10-22","cited_by_count":13,"area":null},{"id":"2512.10938","title":"Stronger Normalization-Free Transformers","created":"2025-12-11","cited_by_count":13,"area":null},{"id":"2106.03743","title":"Proxy-Normalizing Activations to Match Batch Normalization while Removing Batch Dependence","created":"2021-06-07","cited_by_count":12,"area":null},{"id":"2104.04946","title":"UniDrop: A Simple yet Effective Technique to Improve Transformer without Extra Cost","created":"2021-04-11","cited_by_count":11,"area":null},{"id":"2201.05842","title":"UDC: Unified DNAS for Compressible TinyML Models","created":"2022-01-15","cited_by_count":10,"area":null},{"id":"2203.00089","title":"Amortized Proximal Optimization","created":"2022-02-28","cited_by_count":9,"area":null},{"id":"2406.01255","title":"On the Nonlinearity of Layer Normalization","created":"2024-06-03","cited_by_count":9,"area":null},{"id":"2306.11680","title":"The Implicit Bias of Batch Normalization in Linear Models and Two-layer Linear Convolutional Neural Networks","created":"2023-06-20","cited_by_count":8,"area":null},{"id":"2507.20057","title":"What Can Grokking Teach Us About Learning Under Nonstationarity?","created":"2025-07-26","cited_by_count":8,"area":null},{"id":"2101.11513","title":"State estimation with limited sensors -- A deep learning based approach","created":"2021-01-27","cited_by_count":7,"area":null},{"id":"2102.11382","title":"Sandwich Batch Normalization: A Drop-In Replacement for Feature Distribution Heterogeneity","created":"2021-02-22","cited_by_count":7,"area":null},{"id":"2312.15681","title":"Partial Fine-Tuning: A Successor to Full Fine-Tuning for Vision Transformers","created":"2023-12-25","cited_by_count":7,"area":null},{"id":"2410.22113","title":"Where Do Large Learning Rates Lead Us?","created":"2024-10-29","cited_by_count":6,"area":null},{"id":"2108.01110","title":"Batch Normalization Preconditioning for Neural Network Training","created":"2021-08-02","cited_by_count":5,"area":null},{"id":"2101.02916","title":"Accelerating Training of Batch Normalization: A Manifold Perspective","created":"2021-01-08","cited_by_count":4,"area":null},{"id":"2211.17180","title":"Nonlinear Advantage: Trained Networks Might Not Be As Complex as You Think","created":"2022-11-30","cited_by_count":4,"area":null},{"id":"2306.00700","title":"On the Weight Dynamics of Deep Normalized Networks","created":"2023-06-01","cited_by_count":4,"area":null},{"id":"2503.02809","title":"A Minimalist Example of Edge-of-Stability and Progressive Sharpening","created":"2025-03-04","cited_by_count":4,"area":null},{"id":"2302.12444","title":"On the Training Instability of Shuffling SGD with Batch Normalization","created":"2023-02-24","cited_by_count":3,"area":null},{"id":"2303.11602","title":"Convergence of variational Monte Carlo simulation and scale-invariant pre-training","created":"2023-03-21","cited_by_count":2,"area":null},{"id":"2102.03497","title":"Weight Rescaling: Effective and Robust Regularization for Deep Neural Networks with Batch Normalization","created":"2021-02-06","cited_by_count":1,"area":null},{"id":"2110.08725","title":"A Riemannian Mean Field Formulation for Two-layer Neural Networks with Batch Normalization","created":"2021-10-17","cited_by_count":1,"area":null},{"id":"2311.11303","title":"Large Learning Rates Improve Generalization: But How Large Are We Talking About?","created":"2023-11-19","cited_by_count":1,"area":null},{"id":"2606.04405","title":"Low-Rank Decay for Grokking in Scale-Invariant Transformers: A Spectral-Geometric View","created":"2026-06-03","cited_by_count":1,"area":null},{"id":"2505.13142","title":"Parallel Layer Normalization for Universal Approximation","created":"2025-05-19","cited_by_count":0,"area":null},{"id":"2510.18934","title":"Position: Many generalization measures for deep learning are fragile","created":"2025-10-21","cited_by_count":0,"area":null},{"id":"2511.00958","title":"The Hidden Power of Normalization Layers in Neural Networks: Exponential Capacity Control","created":"2025-11-02","cited_by_count":0,"area":null},{"id":"2511.07308","title":"Can Stationary Distributions of Scale-Invariant Neural Networks Be Described by the Thermodynamics of an Ideal Gas?","created":"2025-11-10","cited_by_count":0,"area":null},{"id":"2604.16809","title":"A Mechanism Study of Delayed Loss Spikes in Batch-Normalized Linear Models","created":"2026-04-18","cited_by_count":0,"area":null},{"id":"2605.29152","title":"Do Deep Networks Forget Initialization? A Forgetting-Time View of Practical Inductive Bias","created":"2026-05-27","cited_by_count":0,"area":null},{"id":"2606.14022","title":"Placing Degree Scales After LayerNorm","created":"2026-06-12","cited_by_count":0,"area":null},{"id":"2606.18303","title":"A Link between Shock-wave Theory and Symmetry-reduced Stochastic Gradient Descent for Artificial Neural Networks","created":"2026-06-16","cited_by_count":0,"area":null},{"id":"2609.09116","title":"When Does Scale-Invariant Optimization Become Unstable? An Exact Schedule Law with Weight Decay","created":"2026-09-08","cited_by_count":0,"area":null}],"next_cursor":null}