{"total":241,"rows":[{"id":"2006.16668","title":"GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding","created":"2020-06-30","cited_by_count":1631,"area":"moe"},{"id":"2102.06701","title":"Explaining Neural Scaling Laws","created":"2021-02-12","cited_by_count":363,"area":null},{"id":"2006.13198","title":"Spectral Bias and Task-Model Alignment Explain Generalization in Kernel Regression and Infinitely Wide Neural Networks","created":"2020-06-23","cited_by_count":166,"area":"alignment"},{"id":"2103.10427","title":"The Low-Rank Simplicity Bias in Deep Networks","created":"2021-03-18","cited_by_count":154,"area":null},{"id":"2205.10487","title":"Scaling Laws and Interpretability of Learning from Repeated Data","created":"2022-05-21","cited_by_count":152,"area":null},{"id":"2210.16859","title":"A Solvable Model of Neural Scaling Laws","created":"2022-10-30","cited_by_count":116,"area":null},{"id":"2109.07740","title":"Scaling Laws for Neural Machine Translation","created":"2021-09-16","cited_by_count":111,"area":null},{"id":"2402.01092","title":"A Dynamical Model of Neural Scaling Laws","created":"2024-02-02","cited_by_count":111,"area":null},{"id":"2205.09653","title":"Self-Consistent Dynamical Field Theory of Kernel Evolution in Wide Neural Networks","created":"2022-05-19","cited_by_count":105,"area":null},{"id":"2309.05858","title":"Uncovering mesa-optimization algorithms in Transformers","created":"2023-09-11","cited_by_count":100,"area":null},{"id":"2105.15004","title":"Generalization Error Rates in Kernel Regression: The Crossover from the Noiseless to Noisy Regime","created":"2021-05-31","cited_by_count":96,"area":null},{"id":"2107.04384","title":"Continual Learning in the Teacher-Student Setup: Impact of Task Similarity","created":"2021-07-09","cited_by_count":88,"area":"distill"},{"id":"2106.15933","title":"Saddle-to-Saddle Dynamics in Deep Linear Networks: Small Initialization Training, Symmetry, and Sparsity","created":"2021-06-30","cited_by_count":85,"area":null},{"id":"2105.14602","title":"On the geometry of generalization and memorization in deep neural networks","created":"2021-05-30","cited_by_count":73,"area":null},{"id":"2103.10948","title":"The Shape of Learning Curves: a Review","created":"2021-03-19","cited_by_count":70,"area":null},{"id":"2109.14119","title":"Stochastic Training is Not Necessary for Generalization","created":"2021-09-29","cited_by_count":63,"area":null},{"id":"2109.02355","title":"A Farewell to the Bias-Variance Tradeoff? An Overview of the Theory of Overparameterized Machine Learning","created":"2021-09-06","cited_by_count":57,"area":null},{"id":"2203.08124","title":"Can Neural Nets Learn the Same Model Twice? Investigating Reproducibility and Double Descent from the Decision Boundary Perspective","created":"2022-03-15","cited_by_count":57,"area":null},{"id":"2210.08571","title":"Dimension free ridge regression","created":"2022-10-16","cited_by_count":51,"area":null},{"id":"2211.11567","title":"Neural networks trained with SGD learn distributions of increasing complexity","created":"2022-11-21","cited_by_count":50,"area":null},{"id":"2405.00592","title":"Scaling and renormalization in high-dimensional regression","created":"2024-05-01","cited_by_count":50,"area":null},{"id":"2102.09972","title":"Implicit Regularization in Tensor Factorization","created":"2021-02-19","cited_by_count":44,"area":null},{"id":"2410.02984","title":"Differentiation and Specialization of Attention Heads via the Refined Local Learning Coefficient","created":"2024-10-03","cited_by_count":40,"area":null},{"id":"2306.04251","title":"Stochastic Collapse: How Gradient Noise Attracts SGD Dynamics Towards Simpler Subnetworks","created":"2023-06-07","cited_by_count":39,"area":null},{"id":"2301.12832","title":"Deep networks for system identification: a Survey","created":"2023-01-30","cited_by_count":37,"area":null},{"id":"2303.14151","title":"Double Descent Demystified: Identifying, Interpreting & Ablating the Sources of a Deep Learning Puzzle","created":"2023-03-24","cited_by_count":37,"area":null},{"id":"2207.06569","title":"Benign, Tempered, or Catastrophic: A Taxonomy of Overfitting","created":"2022-07-14","cited_by_count":36,"area":null},{"id":"2107.06608","title":"Continuous vs. Discrete Optimization of Deep Neural Networks","created":"2021-07-14","cited_by_count":34,"area":null},{"id":"2111.13293","title":"KNAS: Green Neural Architecture Search","created":"2021-11-26","cited_by_count":33,"area":null},{"id":"2105.11686","title":"Towards Understanding the Condensation of Neural Networks at Initial Training","created":"2021-05-25","cited_by_count":32,"area":null},{"id":"2407.03475","title":"How JEPA Avoids Noisy Features: The Implicit Bias of Deep Linear Self Distillation Networks","created":"2024-07-03","cited_by_count":31,"area":"distill"},{"id":"2308.12553","title":"Don't blame Dataset Shift! Shortcut Learning due to Gradients and Cross Entropy","created":"2023-08-24","cited_by_count":30,"area":null},{"id":"2202.00565","title":"Data-driven emergence of convolutional structure in neural networks","created":"2022-02-01","cited_by_count":29,"area":null},{"id":"2310.06301","title":"Dynamical versus Bayesian Phase Transitions in a Toy Model of Superposition","created":"2023-10-10","cited_by_count":29,"area":"mechinterp"},{"id":"2604.21691","title":"There Will Be a Scientific Theory of Deep Learning","created":"2026-04-23","cited_by_count":29,"area":null},{"id":"2106.08068","title":"An Analytical Theory of Curriculum Learning in Teacher-Student Networks","created":"2021-06-15","cited_by_count":27,"area":"distill"},{"id":"2309.14563","title":"Towards a statistical theory of data selection under weak supervision","created":"2023-09-25","cited_by_count":27,"area":null},{"id":"2105.14301","title":"A Theory of Neural Tangent Kernel Alignment and Its Influence on Training","created":"2021-05-29","cited_by_count":25,"area":null},{"id":"2106.05418","title":"Probing transfer learning with a model of synthetic correlated datasets","created":"2021-06-09","cited_by_count":25,"area":null},{"id":"2505.16959","title":"Bigger Isn't Always Memorizing: Early Stopping Overparameterized Diffusion Models","created":"2025-05-22","cited_by_count":25,"area":"diffusion"},{"id":"2112.03215","title":"Multi-scale Feature Learning Dynamics: Insights for Double Descent","created":"2021-12-06","cited_by_count":24,"area":null},{"id":"2203.00573","title":"Contrasting random and learned features in deep Bayesian linear regression","created":"2022-03-01","cited_by_count":24,"area":null},{"id":"2410.19637","title":"A distributional simplicity bias in the learning dynamics of transformers","created":"2024-10-25","cited_by_count":24,"area":null},{"id":"2505.19227","title":"Scaling Laws for Gradient Descent and Sign Descent for Linear Bigram Models under Zipf's Law","created":"2025-05-25","cited_by_count":24,"area":null},{"id":"2102.07227","title":"Learning by Turning: Neural Architecture Aware Optimisation","created":"2021-02-14","cited_by_count":23,"area":null},{"id":"2401.05821","title":"Interpretable Concept Bottlenecks to Align Reinforcement Learning Agents","created":"2024-01-11","cited_by_count":23,"area":"deeprl"},{"id":"2405.05847","title":"Learned feature representations are biased by complexity, learning order, position, and more","created":"2024-05-09","cited_by_count":23,"area":null},{"id":"2205.14846","title":"Precise Learning Curves and Higher-Order Scaling Limits for Dot Product Kernel Regression","created":"2022-05-30","cited_by_count":22,"area":null},{"id":"2108.12006","title":"When and how epochwise double descent happens","created":"2021-08-26","cited_by_count":21,"area":null},{"id":"2204.06274","title":"Overparameterized Linear Regression under Adversarial Attacks","created":"2022-04-13","cited_by_count":21,"area":null},{"id":"2212.14457","title":"Bayesian Interpolation with Deep Linear Networks","created":"2022-12-29","cited_by_count":21,"area":null},{"id":"2405.03676","title":"Why is SAM Robust to Label Noise?","created":"2024-05-06","cited_by_count":21,"area":null},{"id":"2502.07551","title":"Early Stopping Against Label Noise Without Validation Data","created":"2025-02-11","cited_by_count":21,"area":null},{"id":"2308.13431","title":"Six Lectures on Linearized Neural Networks","created":"2023-08-25","cited_by_count":20,"area":null},{"id":"2201.13383","title":"Fluctuations, Bias, Variance & Ensemble of Learners: Exact Asymptotics for Convex Losses in High-Dimension","created":"2022-01-31","cited_by_count":19,"area":null},{"id":"2312.00935","title":"Understanding Unimodal Bias in Multimodal Deep Linear Networks","created":"2023-12-01","cited_by_count":19,"area":null},{"id":"2405.20236","title":"Disentangling and Mitigating the Impact of Task Similarity for Continual Learning","created":"2024-05-30","cited_by_count":19,"area":null},{"id":"2502.02531","title":"Deep Linear Network Training Dynamics from Random Initialization: Data, Width, Depth, and Hyperparameter Transfer","created":"2025-02-04","cited_by_count":19,"area":null},{"id":"2405.17580","title":"Mixed Dynamics In Linear Networks: Unifying the Lazy and Active Regimes","created":"2024-05-27","cited_by_count":18,"area":null},{"id":"2510.14807","title":"Beyond the Sampled Token: Preserving Candidate Support in RLVR","created":"2025-10-16","cited_by_count":18,"area":null},{"id":"2101.08732","title":"Self-Adaptive Training: Bridging Supervised and Self-Supervised Learning","created":"2021-01-21","cited_by_count":17,"area":null},{"id":"2110.11805","title":"Model, sample, and epoch-wise descents: exact solution of gradient flow in the random feature model","created":"2021-10-22","cited_by_count":17,"area":null},{"id":"2201.12655","title":"Error Scaling Laws for Kernel Classification under Source and Capacity Conditions","created":"2022-01-29","cited_by_count":17,"area":null},{"id":"2202.05808","title":"Investigating Power laws in Deep Representation Learning","created":"2022-02-11","cited_by_count":17,"area":null},{"id":"2405.13456","title":"Deep linear networks for regression are implicitly regularized towards flat minima","created":"2024-05-22","cited_by_count":17,"area":null},{"id":"2507.12224","title":"Optimizers Qualitatively Alter Solutions And We Should Leverage This","created":"2025-07-16","cited_by_count":17,"area":null},{"id":"2510.01098","title":"Theory of Scaling Laws for In-Context Regression: Depth, Width, Context and Time","created":"2025-10-01","cited_by_count":16,"area":null},{"id":"2310.08513","title":"How connectivity structure shapes rich and lazy learning in neural circuits","created":"2023-10-12","cited_by_count":15,"area":null},{"id":"2407.06076","title":"Understanding Visual Feature Reliance through the Lens of Complexity","created":"2024-07-08","cited_by_count":15,"area":null},{"id":"2512.20607","title":"Saddle-to-Saddle Dynamics Explains A Simplicity Bias Across Neural Network Architectures","created":"2025-12-23","cited_by_count":15,"area":null},{"id":"2102.08373","title":"Analysis of feature learning in weight-tied autoencoders via the mean field lens","created":"2021-02-16","cited_by_count":14,"area":null},{"id":"2107.11413","title":"An Instance-Dependent Simulation Framework for Learning with Label Noise","created":"2021-07-23","cited_by_count":14,"area":null},{"id":"2402.00152","title":"Deeper or Wider: A Perspective from Optimal Generalization Error with Sobolev Loss","created":"2024-01-31","cited_by_count":14,"area":null},{"id":"2502.03350","title":"Optimal Task Order for Continual Learning of Multiple Tasks","created":"2025-02-05","cited_by_count":14,"area":null},{"id":"2101.06967","title":"On Data-Augmentation and Consistency-Based Semi-Supervised Learning","created":"2021-01-18","cited_by_count":13,"area":null},{"id":"2103.04554","title":"Exact Gap between Generalization Error and Uniform Convergence in Random Feature Models","created":"2021-03-08","cited_by_count":13,"area":null},{"id":"2103.05243","title":"On the Generalization Power of Overfitted Two-Layer Neural Tangent Kernel Models","created":"2021-03-09","cited_by_count":13,"area":null},{"id":"2103.05524","title":"On the interplay between data structure and loss function in classification problems","created":"2021-03-09","cited_by_count":13,"area":null},{"id":"2201.04753","title":"Largest Eigenvalues of the Conjugate Kernel of Single-Layered Neural Networks","created":"2022-01-13","cited_by_count":13,"area":null},{"id":"2303.01475","title":"Over-training with Mixup May Hurt Generalization","created":"2023-03-02","cited_by_count":13,"area":null},{"id":"2502.07998","title":"Adaptive kernel predictors from feature-learning infinite limits of neural networks","created":"2025-02-11","cited_by_count":13,"area":null},{"id":"2105.02468","title":"Relative stability toward diffeomorphisms indicates performance in deep nets","created":"2021-05-06","cited_by_count":12,"area":null},{"id":"2106.08414","title":"On the Sample Complexity and Metastability of Heavy-tailed Policy Search in Continuous Control","created":"2021-06-15","cited_by_count":12,"area":null},{"id":"2108.00774","title":"A Random Matrix Perspective on Random Tensors","created":"2021-08-02","cited_by_count":12,"area":null},{"id":"2203.07337","title":"Phenomenology of Double Descent in Finite-Width Neural Networks","created":"2022-03-14","cited_by_count":12,"area":null},{"id":"2210.17449","title":"Globally Gated Deep Linear Networks","created":"2022-10-31","cited_by_count":12,"area":null},{"id":"2307.07785","title":"The Interpolating Information Criterion for Overparameterized Models","created":"2023-07-15","cited_by_count":12,"area":null},{"id":"2405.14689","title":"Cascade of phase transitions in the training of Energy-based models","created":"2024-05-23","cited_by_count":12,"area":null},{"id":"2102.00940","title":"Meta-learning with negative learning rates","created":"2021-02-01","cited_by_count":11,"area":null},{"id":"2302.01952","title":"On a continuous time model of gradient descent dynamics and instability in deep learning","created":"2023-02-03","cited_by_count":11,"area":null},{"id":"2406.04425","title":"On Regularization via Early Stopping for Least Squares Regression","created":"2024-06-06","cited_by_count":11,"area":null},{"id":"2107.09437","title":"Edge of chaos as a guiding principle for modern neural network training","created":"2021-07-20","cited_by_count":10,"area":null},{"id":"2202.07184","title":"On the Origins of the Block Structure Phenomenon in Neural Network Representations","created":"2022-02-15","cited_by_count":10,"area":null},{"id":"2302.07557","title":"On the Generalization of PINNs outside the training domain and the Hyperparameters influencing it","created":"2023-02-15","cited_by_count":10,"area":null},{"id":"2404.08921","title":"PNeRV: Enhancing Spatial Consistency via Pyramidal Neural Representation for Videos","created":"2024-04-13","cited_by_count":10,"area":null},{"id":"2603.00742","title":"To Use or not to Use Muon: How Simplicity Bias in Optimizers Matters","created":"2026-02-28","cited_by_count":10,"area":null},{"id":"2106.03485","title":"Redundant representations help generalization in wide neural networks","created":"2021-06-07","cited_by_count":9,"area":null},{"id":"2208.07951","title":"On the generalization of learning algorithms that do not converge","created":"2022-08-16","cited_by_count":9,"area":null},{"id":"2302.05059","title":"Effects of noise on the overparametrization of quantum neural networks","created":"2023-02-10","cited_by_count":9,"area":null},{"id":"2311.04787","title":"Why Do Probabilistic Clinical Models Fail To Transport Between Sites?","created":"2023-11-08","cited_by_count":9,"area":null}],"next_cursor":"100"}