{"total":62,"rows":[{"id":"2312.02696","title":"Analyzing and Improving the Training Dynamics of Diffusion Models","created":"2023-12-05","cited_by_count":402,"area":"diffusion"},{"id":"2108.01624","title":"Large-Scale Differentially Private BERT","created":"2021-08-03","cited_by_count":118,"area":null},{"id":"2202.05306","title":"Characterizing and overcoming the greedy nature of learning in multi-modal deep neural networks","created":"2022-02-10","cited_by_count":95,"area":null},{"id":"2210.09221","title":"Vision Transformers provably learn spatial structure","created":"2022-10-13","cited_by_count":93,"area":null},{"id":"2206.07085","title":"Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction","created":"2022-06-14","cited_by_count":86,"area":null},{"id":"2310.04415","title":"Why Do We Need Weight Decay in Modern Deep Learning?","created":"2023-10-06","cited_by_count":72,"area":null},{"id":"2102.04761","title":"Quasi-Global Momentum: Accelerating Decentralized Deep Learning on Heterogeneous Data","created":"2021-02-09","cited_by_count":69,"area":null},{"id":"2105.05246","title":"Spectral Normalisation for Deep Reinforcement Learning: an Optimisation Perspective","created":"2021-05-11","cited_by_count":65,"area":null},{"id":"2105.01029","title":"Initialization and Regularization of Factorized Neural Layers","created":"2021-05-03","cited_by_count":58,"area":"distill"},{"id":"2404.04454","title":"Implicit Bias of AdamW: $\\ell_\\infty$ Norm Constrained Optimization","created":"2024-04-05","cited_by_count":58,"area":null},{"id":"2407.01800","title":"Normalization and effective learning rates in reinforcement learning","created":"2024-07-01","cited_by_count":58,"area":"deeprl"},{"id":"2305.17212","title":"Rotational Equilibrium: How Weight Decay Balances Learning Across Neural Networks","created":"2023-05-26","cited_by_count":53,"area":null},{"id":"2105.02716","title":"Noether's Learning Dynamics: Role of Symmetry Breaking in Neural Networks","created":"2021-05-06","cited_by_count":50,"area":null},{"id":"2303.15725","title":"Solving Regularized Exp, Cosh and Sinh Regression Problems","created":"2023-03-28","cited_by_count":44,"area":null},{"id":"2505.13738","title":"Power Lines: Scaling Laws for Weight Decay and Batch Size in LLM Pre-training","created":"2025-05-19","cited_by_count":43,"area":null},{"id":"2207.05931","title":"Towards understanding how momentum improves generalization in deep learning","created":"2022-07-13","cited_by_count":40,"area":null},{"id":"2202.00980","title":"Robust Training of Neural Networks Using Scale Invariant Architectures","created":"2022-02-02","cited_by_count":29,"area":null},{"id":"2206.08900","title":"Adapting the Linearised Laplace Model Evidence for Modern Deep Learning","created":"2022-06-17","cited_by_count":29,"area":null},{"id":"2110.12661","title":"ZerO Initialization: Initializing Neural Networks with only Zeros and Ones","created":"2021-10-25","cited_by_count":28,"area":null},{"id":"2106.15739","title":"On the Periodic Behavior of Neural Network Training with Batch Normalization and Weight Decay","created":"2021-06-29","cited_by_count":20,"area":null},{"id":"2209.03695","title":"Training Scale-Invariant Neural Networks on the Sphere Can Happen in Three Regimes","created":"2022-09-08","cited_by_count":19,"area":null},{"id":"2303.11135","title":"TWINS: A Fine-Tuning Framework for Improved Transferability of Adversarial Robustness and Generalization","created":"2023-03-20","cited_by_count":18,"area":null},{"id":"2205.11921","title":"Compression-aware Training of Neural Networks using Frank-Wolfe","created":"2022-05-24","cited_by_count":16,"area":null},{"id":"2606.16899","title":"Fantastic Pretraining Optimizers and Where to Find Them II: Hyperball Optimization","created":"2026-06-15","cited_by_count":16,"area":null},{"id":"2210.15898","title":"Toward Equation of Motion for Deep Neural Networks: Continuous-time Gradient Descent and Discretization Error Analysis","created":"2022-10-28","cited_by_count":13,"area":null},{"id":"2106.03743","title":"Proxy-Normalizing Activations to Match Batch Normalization while Removing Batch Dependence","created":"2021-06-07","cited_by_count":12,"area":null},{"id":"2102.04270","title":"Enabling Binary Neural Network Training on the Edge","created":"2021-02-08","cited_by_count":11,"area":null},{"id":"2203.00089","title":"Amortized Proximal Optimization","created":"2022-02-28","cited_by_count":9,"area":null},{"id":"2406.01255","title":"On the Nonlinearity of Layer Normalization","created":"2024-06-03","cited_by_count":9,"area":null},{"id":"2306.11680","title":"The Implicit Bias of Batch Normalization in Linear Models and Two-layer Linear Convolutional Neural Networks","created":"2023-06-20","cited_by_count":8,"area":null},{"id":"2303.02095","title":"Data-Efficient Training of CNNs and Transformers with Coresets: A Stability Perspective","created":"2023-03-03","cited_by_count":6,"area":null},{"id":"2312.10359","title":"Conformer-Based Speech Recognition On Extreme Edge-Computing Devices","created":"2023-12-16","cited_by_count":6,"area":null},{"id":"2306.00700","title":"On the Weight Dynamics of Deep Normalized Networks","created":"2023-06-01","cited_by_count":4,"area":null},{"id":"2407.04797","title":"Revealing the Utilized Rank of Subspaces of Learning in Neural Networks","created":"2024-07-05","cited_by_count":4,"area":null},{"id":"2607.22444","title":"Hyperball May Not Be a Free Lunch","created":"2026-07-24","cited_by_count":4,"area":null},{"id":"2103.15345","title":"FixNorm: Dissecting Weight Decay for Training Deep Neural Networks","created":"2021-03-29","cited_by_count":3,"area":null},{"id":"2606.23637","title":"Muown Implicitly Performs Angular Step-size Decay","created":"2026-06-22","cited_by_count":3,"area":null},{"id":"2606.25971","title":"Improving Neural Network Training by Decoupling the Magnitude and Direction of Weight Vectors","created":"2026-06-24","cited_by_count":3,"area":"moe"},{"id":"2608.24814","title":"Effective Learning Rate Governs Loss Dynamics in Language Model Pretraining","created":"2026-08-25","cited_by_count":3,"area":null},{"id":"2210.13599","title":"Noise Injection as a Probe of Deep Learning Dynamics","created":"2022-10-24","cited_by_count":2,"area":null},{"id":"2211.08544","title":"Exploiting the Partly Scratch-off Lottery Ticket for Quantization-Aware Training","created":"2022-11-12","cited_by_count":2,"area":null},{"id":"2404.19248","title":"Scheduling Weight Transitions for Quantization-Aware Training","created":"2024-04-30","cited_by_count":2,"area":null},{"id":"2510.04686","title":"How does the optimizer implicitly bias the model merging loss landscape?","created":"2025-10-06","cited_by_count":2,"area":null},{"id":"2606.26749","title":"Structure Before Collapse: Transient semantic geometry in next-token prediction","created":"2026-06-25","cited_by_count":2,"area":null},{"id":"2606.29158","title":"On the Nonlinearity of Learning Rate Scaling for LLM Training","created":"2026-06-28","cited_by_count":2,"area":null},{"id":"2102.03497","title":"Weight Rescaling: Effective and Robust Regularization for Deep Neural Networks with Batch Normalization","created":"2021-02-06","cited_by_count":1,"area":null},{"id":"2205.07260","title":"Guidelines for the Regularization of Gammas in Batch Normalization for Deep Residual Networks","created":"2022-05-15","cited_by_count":1,"area":null},{"id":"2309.01694","title":"No Data Augmentation? Alternative Regularizations for Effective Training on Small Datasets","created":"2023-09-04","cited_by_count":1,"area":null},{"id":"2504.15991","title":"Efficient Adaptation of Deep Neural Networks for Semantic Segmentation in Space Applications","created":"2025-04-22","cited_by_count":1,"area":null},{"id":"2605.24420","title":"Batch Normalization Amplifies Memorization and Privacy Risks","created":"2026-05-23","cited_by_count":1,"area":null},{"id":"2607.21005","title":"Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay","created":"2026-07-23","cited_by_count":1,"area":null},{"id":"2101.01178","title":"Advances in Electron Microscopy with Deep Learning","created":"2021-01-04","cited_by_count":0,"area":null},{"id":"2105.13464","title":"Training With Data Dependent Dynamic Learning Rates","created":"2021-05-27","cited_by_count":0,"area":null},{"id":"2108.05839","title":"Logit Attenuating Weight Normalization","created":"2021-08-12","cited_by_count":0,"area":null},{"id":"2203.08174","title":"Towards understanding deep learning with the natural clustering prior","created":"2022-03-15","cited_by_count":0,"area":null},{"id":"2207.02625","title":"$L_2$BN: Enhancing Batch Normalization by Equalizing the $L_2$ Norms of Features","created":"2022-07-06","cited_by_count":0,"area":null},{"id":"2211.02686","title":"LightNorm: Area and Energy-Efficient Batch Normalization Hardware for On-Device DNN Training","created":"2022-11-04","cited_by_count":0,"area":null},{"id":"2511.07308","title":"Can Stationary Distributions of Scale-Invariant Neural Networks Be Described by the Thermodynamics of an Ideal Gas?","created":"2025-11-10","cited_by_count":0,"area":null},{"id":"2604.16809","title":"A Mechanism Study of Delayed Loss Spikes in Batch-Normalized Linear Models","created":"2026-04-18","cited_by_count":0,"area":null},{"id":"2605.09154","title":"Predicting Large Model Test Losses with a Noisy Quadratic System","created":"2026-05-09","cited_by_count":0,"area":null},{"id":"2608.21024","title":"RODE: A Radial-Orthogonal Decoupled Engine for Optimization","created":"2026-08-21","cited_by_count":0,"area":null},{"id":"2609.09116","title":"When Does Scale-Invariant Optimization Become Unstable? An Exact Schedule Law with Weight Decay","created":"2026-09-08","cited_by_count":0,"area":null}],"next_cursor":null}