{"total":113,"rows":[{"id":"2204.06745","title":"GPT-NeoX-20B: An Open-Source Autoregressive Language Model","created":"2022-04-14","cited_by_count":713,"area":null},{"id":"2312.02696","title":"Analyzing and Improving the Training Dynamics of Diffusion Models","created":"2023-12-05","cited_by_count":402,"area":"diffusion"},{"id":"2210.01117","title":"Omnigrok: Grokking Beyond Algorithmic Data","created":"2022-10-03","cited_by_count":146,"area":null},{"id":"2309.02390","title":"Explaining grokking through circuit efficiency","created":"2023-09-05","cited_by_count":111,"area":null},{"id":"2104.04975","title":"Scalable Marginal Likelihood Estimation for Model Selection in Deep Learning","created":"2021-04-11","cited_by_count":106,"area":null},{"id":"2202.05306","title":"Characterizing and overcoming the greedy nature of learning in multi-modal deep neural networks","created":"2022-02-10","cited_by_count":95,"area":null},{"id":"2206.07085","title":"Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction","created":"2022-06-14","cited_by_count":86,"area":null},{"id":"2310.04415","title":"Why Do We Need Weight Decay in Modern Deep Learning?","created":"2023-10-06","cited_by_count":72,"area":null},{"id":"2102.04761","title":"Quasi-Global Momentum: Accelerating Decentralized Deep Learning on Heterogeneous Data","created":"2021-02-09","cited_by_count":69,"area":null},{"id":"2103.16725","title":"SimPLE: Similar Pseudo Label Exploitation for Semi-Supervised Classification","created":"2021-03-30","cited_by_count":68,"area":null},{"id":"2210.05337","title":"SGD with Large Step Sizes Learns Sparse Features","created":"2022-10-11","cited_by_count":68,"area":null},{"id":"2105.01029","title":"Initialization and Regularization of Factorized Neural Layers","created":"2021-05-03","cited_by_count":58,"area":"distill"},{"id":"2302.10911","title":"Revisiting Weighted Aggregation in Federated Learning with Neural Networks","created":"2023-02-14","cited_by_count":58,"area":null},{"id":"2404.04454","title":"Implicit Bias of AdamW: $\\ell_\\infty$ Norm Constrained Optimization","created":"2024-04-05","cited_by_count":58,"area":null},{"id":"2305.17212","title":"Rotational Equilibrium: How Weight Decay Balances Learning Across Neural Networks","created":"2023-05-26","cited_by_count":53,"area":null},{"id":"2105.02716","title":"Noether's Learning Dynamics: Role of Symmetry Breaking in Neural Networks","created":"2021-05-06","cited_by_count":50,"area":null},{"id":"2405.13698","title":"How to set AdamW's weight decay as you scale model and dataset size","created":"2024-05-22","cited_by_count":43,"area":null},{"id":"2505.13738","title":"Power Lines: Scaling Laws for Weight Decay and Batch Size in LLM Pre-training","created":"2025-05-19","cited_by_count":43,"area":null},{"id":"2408.11804","title":"Approaching Deep Learning through the Spectral Dynamics of Weights","created":"2024-08-21","cited_by_count":31,"area":null},{"id":"2201.12250","title":"Gradient Descent on Neurons and its Link to Approximate Second-Order Optimization","created":"2022-01-28","cited_by_count":30,"area":null},{"id":"2202.00980","title":"Robust Training of Neural Networks Using Scale Invariant Architectures","created":"2022-02-02","cited_by_count":29,"area":null},{"id":"2410.23819","title":"Weight decay induces low-rank attention layers","created":"2024-10-31","cited_by_count":27,"area":null},{"id":"2404.00851","title":"Prompt Learning via Meta-Regularization","created":"2024-04-01","cited_by_count":26,"area":"mllm"},{"id":"2310.16441","title":"Grokking in Linear Estimators -- A Solvable Model that Groks without Understanding","created":"2023-10-25","cited_by_count":25,"area":null},{"id":"2106.02925","title":"Tensor Normal Training for Deep Learning Models","created":"2021-06-05","cited_by_count":24,"area":null},{"id":"2305.07583","title":"MoMo: Momentum Models for Adaptive Learning Rates","created":"2023-05-12","cited_by_count":24,"area":null},{"id":"2409.15156","title":"Rethinking Conventional Wisdom in Machine Learning: From Generalization to Scaling","created":"2024-09-23","cited_by_count":23,"area":"lr"},{"id":"2506.02285","title":"Why Gradients Rapidly Increase Near the End of Training","created":"2025-06-02","cited_by_count":22,"area":null},{"id":"2106.15739","title":"On the Periodic Behavior of Neural Network Training with Batch Normalization and Weight Decay","created":"2021-06-29","cited_by_count":20,"area":null},{"id":"2303.12922","title":"Revisiting the Fragility of Influence Functions","created":"2023-03-22","cited_by_count":20,"area":null},{"id":"2510.19093","title":"Weight Decay may matter more than muP for Learning Rate Transfer in Practice","created":"2025-10-21","cited_by_count":20,"area":null},{"id":"2209.03695","title":"Training Scale-Invariant Neural Networks on the Sphere Can Happen in Three Regimes","created":"2022-09-08","cited_by_count":19,"area":null},{"id":"2606.16899","title":"Fantastic Pretraining Optimizers and Where to Find Them II: Hyperball Optimization","created":"2026-06-15","cited_by_count":16,"area":null},{"id":"2301.04935","title":"A Stochastic Proximal Polyak Step Size","created":"2023-01-12","cited_by_count":15,"area":null},{"id":"2312.12226","title":"On the Parameterization of Second-Order Optimization Effective Towards the Infinite Width","created":"2023-12-19","cited_by_count":15,"area":null},{"id":"2202.08325","title":"A Data-Augmentation Is Worth A Thousand Samples: Exact Quantification From Analytical Augmented Sample Moments","created":"2022-02-16","cited_by_count":14,"area":null},{"id":"2410.03249","title":"How Much Can We Forget about Data Contamination?","created":"2024-10-04","cited_by_count":14,"area":null},{"id":"2511.03202","title":"Provable Separations between Memorization and Generalization in Diffusion Models","created":"2025-11-05","cited_by_count":14,"area":"diffusion"},{"id":"2210.00094","title":"Improving Robustness with Adaptive Weight Decay","created":"2022-09-30","cited_by_count":13,"area":null},{"id":"2210.15898","title":"Toward Equation of Motion for Deep Neural Networks: Continuous-time Gradient Descent and Discretization Error Analysis","created":"2022-10-28","cited_by_count":13,"area":null},{"id":"2106.03743","title":"Proxy-Normalizing Activations to Match Batch Normalization while Removing Batch Dependence","created":"2021-06-07","cited_by_count":12,"area":null},{"id":"2206.05658","title":"Improving Pre-trained Language Model Fine-tuning with Noise Stability Regularization","created":"2022-06-12","cited_by_count":12,"area":null},{"id":"2203.06832","title":"Semi-Discrete Normalizing Flows through Differentiable Tessellation","created":"2022-03-14","cited_by_count":11,"area":null},{"id":"2107.09437","title":"Edge of chaos as a guiding principle for modern neural network training","created":"2021-07-20","cited_by_count":10,"area":null},{"id":"2306.03968","title":"Stochastic Marginal Likelihood Gradients using Neural Tangent Kernels","created":"2023-06-06","cited_by_count":10,"area":null},{"id":"2308.07163","title":"HyperSparse Neural Networks: Shifting Exploration to Exploitation through Adaptive Regularization","created":"2023-08-14","cited_by_count":9,"area":null},{"id":"2406.01255","title":"On the Nonlinearity of Layer Normalization","created":"2024-06-03","cited_by_count":9,"area":null},{"id":"2202.04124","title":"A Mini-Block Fisher Method for Deep Neural Networks","created":"2022-02-08","cited_by_count":8,"area":null},{"id":"2202.08978","title":"Cyclical Focal Loss","created":"2022-02-16","cited_by_count":8,"area":null},{"id":"2209.15208","title":"Scale-invariant Bayesian Neural Networks with Connectivity Tangent Kernel","created":"2022-09-30","cited_by_count":7,"area":null},{"id":"2305.16573","title":"Exploring Weight Balancing on Long-Tailed Recognition Problem","created":"2023-05-26","cited_by_count":7,"area":null},{"id":"2309.03004","title":"A Theoretical Explanation of Activation Sparsity through Flat Minima and Adversarial Robustness","created":"2023-09-06","cited_by_count":7,"area":null},{"id":"2410.12707","title":"FusionLLM: A Decentralized LLM Training System on Geo-distributed GPUs with Adaptive Compression","created":"2024-10-16","cited_by_count":7,"area":null},{"id":"2102.06737","title":"Kronecker-factored Quasi-Newton Methods for Deep Learning","created":"2021-02-12","cited_by_count":6,"area":null},{"id":"2103.06733","title":"Intraclass clustering: an implicit learning ability that regularizes DNNs","created":"2021-03-11","cited_by_count":6,"area":null},{"id":"2306.06098","title":"Error Feedback Can Accurately Compress Preconditioners","created":"2023-06-09","cited_by_count":6,"area":null},{"id":"2603.11970","title":"Statistical and structural identifiability in representation learning","created":"2026-03-12","cited_by_count":6,"area":null},{"id":"2107.10523","title":"Target-Oriented Fine-tuning for Zero-Resource Named Entity Recognition","created":"2021-07-22","cited_by_count":5,"area":null},{"id":"2202.08835","title":"General Cyclical Training of Neural Networks","created":"2022-02-17","cited_by_count":5,"area":null},{"id":"2305.15912","title":"Neural Characteristic Activation Analysis and Geometric Parameterization for ReLU Networks","created":"2023-05-25","cited_by_count":5,"area":null},{"id":"2311.09058","title":"Improving Deep Learning Optimization through Constrained Parameter Regularization","created":"2023-11-15","cited_by_count":5,"area":null},{"id":"2408.09414","title":"Clustering and Alignment: Understanding the Training Dynamics in Modular Addition","created":"2024-08-18","cited_by_count":5,"area":null},{"id":"2502.14344","title":"Towards Accurate Binary Spiking Neural Networks: Learning with Adaptive Gradient Modulation Mechanism","created":"2025-02-20","cited_by_count":5,"area":null},{"id":"2103.08802","title":"Parareal Neural Networks Emulating a Parallel-in-time Algorithm","created":"2021-03-16","cited_by_count":4,"area":null},{"id":"2407.04797","title":"Revealing the Utilized Rank of Subspaces of Learning in Neural Networks","created":"2024-07-05","cited_by_count":4,"area":null},{"id":"2410.11985","title":"The Fair Language Model Paradox","created":"2024-10-15","cited_by_count":4,"area":null},{"id":"2410.14602","title":"Data Diversity as Implicit Regularization: How Does Diversity Shape the Weight Space of Deep Neural Networks?","created":"2024-10-18","cited_by_count":4,"area":null},{"id":"2509.25049","title":"Efficient Hyperparameter Tuning via Trajectory Invariance Principle","created":"2025-09-29","cited_by_count":4,"area":null},{"id":"2602.01642","title":"The Effect of Mini-Batch Noise on the Implicit Bias of Adam","created":"2026-02-02","cited_by_count":4,"area":null},{"id":"2602.11137","title":"Weight Decay Improves Language Model Plasticity","created":"2026-02-11","cited_by_count":4,"area":null},{"id":"2607.22444","title":"Hyperball May Not Be a Free Lunch","created":"2026-07-24","cited_by_count":4,"area":null},{"id":"2103.15345","title":"FixNorm: Dissecting Weight Decay for Training Deep Neural Networks","created":"2021-03-29","cited_by_count":3,"area":null},{"id":"2507.13089","title":"GLAD: Generalizable Tuning for Vision-Language Models","created":"2025-07-17","cited_by_count":3,"area":"mllm"},{"id":"2605.25608","title":"Learning Sparse Compositional Functions with Norm-Constrained Neural Networks","created":"2026-05-25","cited_by_count":3,"area":null},{"id":"2608.24814","title":"Effective Learning Rate Governs Loss Dynamics in Language Model Pretraining","created":"2026-08-25","cited_by_count":3,"area":null},{"id":"2102.08183","title":"Comparison of semi-supervised deep learning algorithms for audio classification","created":"2021-02-16","cited_by_count":2,"area":null},{"id":"2202.10788","title":"Explicit Regularization via Regularizer Mirror Descent","created":"2022-02-22","cited_by_count":2,"area":null},{"id":"2209.09240","title":"Distributed Semi-supervised Fuzzy Regression with Interpolation Consistency Regularization","created":"2022-09-18","cited_by_count":2,"area":null},{"id":"2209.11436","title":"Understanding Open-Set Recognition by Jacobian Norm and Inter-Class Separation","created":"2022-09-23","cited_by_count":2,"area":null},{"id":"2601.19597","title":"The Geometric Mechanics of Contrastive Representation Learning: Alignment Potentials, Entropic Dispersion, and Cross-modal Divergence","created":"2026-01-27","cited_by_count":2,"area":null},{"id":"2602.16740","title":"Quantifying LLM Attention-Head Stability: Implications for Circuit Universality","created":"2026-02-17","cited_by_count":2,"area":"mechinterp"},{"id":"2102.00632","title":"ConvNets for Counting: Object Detection of Transient Phenomena in Steelpan Drums","created":"2021-02-01","cited_by_count":1,"area":null},{"id":"2102.03497","title":"Weight Rescaling: Effective and Robust Regularization for Deep Neural Networks with Batch Normalization","created":"2021-02-06","cited_by_count":1,"area":null},{"id":"2205.07260","title":"Guidelines for the Regularization of Gammas in Batch Normalization for Deep Residual Networks","created":"2022-05-15","cited_by_count":1,"area":null},{"id":"2210.15764","title":"Noise Injection Node Regularization for Robust Learning","created":"2022-10-27","cited_by_count":1,"area":null},{"id":"2302.03193","title":"On the Ideal Number of Groups for Isometric Gradient Propagation","created":"2023-02-07","cited_by_count":1,"area":null},{"id":"2309.01694","title":"No Data Augmentation? Alternative Regularizations for Effective Training on Small Datasets","created":"2023-09-04","cited_by_count":1,"area":null},{"id":"2402.11224","title":"Neural Networks with (Low-Precision) Polynomial Approximations: New Insights and Techniques for Accuracy Improvement","created":"2024-02-17","cited_by_count":1,"area":null},{"id":"2411.00383","title":"Preventing Model Collapse in Deep Canonical Correlation Analysis by Noise Regularization","created":"2024-11-01","cited_by_count":1,"area":"synth"},{"id":"2507.04683","title":"Recovering Plasticity of Neural Networks via Soft Weight Rescaling","created":"2025-07-07","cited_by_count":1,"area":null},{"id":"2512.08217","title":"Correction of Decoupled Weight Decay","created":"2025-12-09","cited_by_count":1,"area":null},{"id":"2602.03879","title":"TruKAN: Towards More Efficient Kolmogorov-Arnold Networks Using Truncated Power Functions","created":"2026-02-02","cited_by_count":1,"area":null},{"id":"2604.07663","title":"SAGE: Sign-Adaptive Gradient for Memory-Efficient LLM Optimization","created":"2026-04-09","cited_by_count":1,"area":null},{"id":"2605.04418","title":"Demystifying Manifold Constraints in LLM Pre-training","created":"2026-05-06","cited_by_count":1,"area":null},{"id":"2607.21005","title":"Weight-norm Criticality: A Mechanism for Loss Spikes Induced by the Normalization and Weight Decay","created":"2026-07-23","cited_by_count":1,"area":null},{"id":"2105.01875","title":"Modulating Regularization Frequency for Efficient Compression-Aware Model Training","created":"2021-05-05","cited_by_count":0,"area":null},{"id":"2105.13464","title":"Training With Data Dependent Dynamic Learning Rates","created":"2021-05-27","cited_by_count":0,"area":null},{"id":"2112.14016","title":"Recursive Least-Squares Estimator-Aided Online Learning for Visual Tracking","created":"2021-12-28","cited_by_count":0,"area":null},{"id":"2202.12378","title":"Deep Learning to advance the Eigenspace Perturbation Method for Turbulence Model Uncertainty Quantification","created":"2022-02-11","cited_by_count":0,"area":null},{"id":"2203.08174","title":"Towards understanding deep learning with the natural clustering prior","created":"2022-03-15","cited_by_count":0,"area":null}],"next_cursor":"100"}