{"total":306,"rows":[{"id":"2205.10343","title":"Towards Understanding Grokking: An Effective Theory of Representation Learning","created":"2022-05-20","cited_by_count":230,"area":null},{"id":"2206.15144","title":"Neural Networks can Learn Representations with Gradient Descent","created":"2022-06-30","cited_by_count":161,"area":null},{"id":"2205.01445","title":"High-dimensional Asymptotics of Feature Learning: How One Gradient Step Improves the Representation","created":"2022-05-03","cited_by_count":150,"area":null},{"id":"2206.06232","title":"Towards Understanding Sharpness-Aware Minimization","created":"2022-06-13","cited_by_count":146,"area":null},{"id":"2206.02618","title":"Generalized Federated Learning via Sharpness Aware Minimization","created":"2022-06-06","cited_by_count":144,"area":null},{"id":"2206.07758","title":"Reconstructing Training Data from Trained Neural Networks","created":"2022-06-15","cited_by_count":133,"area":null},{"id":"2105.14368","title":"Fit without fear: remarkable mathematical phenomena of deep learning through the prism of interpolation","created":"2021-05-29","cited_by_count":132,"area":null},{"id":"2203.12221","title":"Modality Competition: What Makes Joint Training of Multi-modal Network Fail in Deep Learning? (Provably)","created":"2022-03-23","cited_by_count":122,"area":null},{"id":"2210.06313","title":"The Lazy Neuron Phenomenon: On Emergence of Activation Sparsity in Transformers","created":"2022-10-12","cited_by_count":122,"area":null},{"id":"2106.06530","title":"Label Noise SGD Provably Prefers Flat Global Minimizers","created":"2021-06-11","cited_by_count":121,"area":null},{"id":"2210.05643","title":"A Kernel-Based View of Language Model Fine-Tuning","created":"2022-10-11","cited_by_count":121,"area":null},{"id":"2106.09524","title":"Implicit Bias of SGD for Diagonal Linear Networks: a Provable Benefit of Stochasticity","created":"2021-06-17","cited_by_count":110,"area":null},{"id":"2110.06914","title":"What Happens after SGD Reaches Zero Loss? --A Mathematical Framework","created":"2021-10-13","cited_by_count":107,"area":null},{"id":"2111.00034","title":"Neural Networks as Kernel Learners: The Silent Alignment Effect","created":"2021-10-29","cited_by_count":106,"area":null},{"id":"2202.14026","title":"Robust Training under Label Noise by Over-parameterization","created":"2022-02-28","cited_by_count":93,"area":null},{"id":"2210.09221","title":"Vision Transformers provably learn spatial structure","created":"2022-10-13","cited_by_count":93,"area":null},{"id":"2210.15651","title":"Learning Single-Index Models with Shallow Neural Networks","created":"2022-10-27","cited_by_count":92,"area":null},{"id":"2302.07011","title":"A Modern Look at the Relationship between Sharpness and Generalization","created":"2023-02-14","cited_by_count":92,"area":null},{"id":"2208.12591","title":"On the Implicit Bias in Deep-Learning Algorithms","created":"2022-08-26","cited_by_count":86,"area":null},{"id":"2106.15933","title":"Saddle-to-Saddle Dynamics in Deep Linear Networks: Small Initialization Training, Symmetry, and Sparsity","created":"2021-06-30","cited_by_count":85,"area":null},{"id":"2210.14199","title":"Same Pre-training Loss, Better Downstream: Implicit Bias Matters for Language Models","created":"2022-10-25","cited_by_count":84,"area":null},{"id":"2110.13905","title":"Gradient Descent on Two-layer Nets: Margin Maximization and Simplicity Bias","created":"2021-10-26","cited_by_count":83,"area":null},{"id":"2311.18817","title":"Dichotomy of Early and Late Phase Implicit Biases Can Provably Induce Grokking","created":"2023-11-30","cited_by_count":80,"area":null},{"id":"2106.15013","title":"Small random initialization is akin to spectral learning: Optimization and generalization guarantees for overparameterized low-rank matrix reconstruction","created":"2021-06-28","cited_by_count":78,"area":null},{"id":"2206.00939","title":"Gradient flow dynamics of shallow ReLU networks for square loss and orthogonal inputs","created":"2022-06-02","cited_by_count":76,"area":null},{"id":"2308.16898","title":"Transformers as Support Vector Machines","created":"2023-08-31","cited_by_count":74,"area":null},{"id":"2102.09769","title":"On the Implicit Bias of Initialization Shape: Beyond Infinitesimal Mirror Descent","created":"2021-02-19","cited_by_count":70,"area":null},{"id":"2210.05337","title":"SGD with Large Step Sizes Learns Sparse Features","created":"2022-10-11","cited_by_count":68,"area":null},{"id":"2112.04716","title":"DR3: Value-Based Deep Reinforcement Learning Requires Explicit Regularization","created":"2021-12-09","cited_by_count":66,"area":"offlinerl"},{"id":"2207.02628","title":"The alignment property of SGD noise and how it helps select flat minima: A stability analysis","created":"2022-07-06","cited_by_count":64,"area":null},{"id":"2306.13596","title":"Max-Margin Token Selection in Attention Mechanism","created":"2023-06-23","cited_by_count":58,"area":null},{"id":"2109.02355","title":"A Farewell to the Bias-Variance Tradeoff? An Overview of the Theory of Overparameterized Machine Learning","created":"2021-09-06","cited_by_count":57,"area":null},{"id":"2304.00488","title":"Saddle-to-Saddle Dynamics in Diagonal Linear Networks","created":"2023-04-02","cited_by_count":57,"area":null},{"id":"2409.17858","title":"How Feature Learning Can Improve Neural Scaling Laws","created":"2024-09-26","cited_by_count":56,"area":null},{"id":"2201.12760","title":"Implicit Regularization Towards Rank Minimization in ReLU Networks","created":"2022-01-30","cited_by_count":53,"area":null},{"id":"2102.12781","title":"Do Input Gradients Highlight Discriminative Features?","created":"2021-02-25","cited_by_count":52,"area":null},{"id":"2306.07042","title":"Transformers learn through gradual rank increase","created":"2023-06-12","cited_by_count":52,"area":null},{"id":"2409.14623","title":"From Lazy to Rich: Exact Learning Dynamics in Deep Linear Networks","created":"2024-09-22","cited_by_count":49,"area":null},{"id":"2406.18214","title":"Trimming the Fat: Efficient Compression of 3D Gaussian Splats through Pruning","created":"2024-06-26","cited_by_count":47,"area":null},{"id":"2501.16265","title":"Training Dynamics of In-Context Learning in Linear Attention","created":"2025-01-27","cited_by_count":46,"area":null},{"id":"2102.09972","title":"Implicit Regularization in Tensor Factorization","created":"2021-02-19","cited_by_count":44,"area":null},{"id":"2305.17490","title":"The Implicit Regularization of Dynamical Stability in Stochastic Gradient Descent","created":"2023-05-27","cited_by_count":44,"area":null},{"id":"2301.11500","title":"Understanding Incremental Learning of Gradient Descent: A Fine-grained Analysis of Matrix Sensing","created":"2023-01-27","cited_by_count":42,"area":null},{"id":"2406.06158","title":"Get rich quick: exact solutions reveal how unbalanced initializations promote rapid feature learning","created":"2024-06-10","cited_by_count":42,"area":null},{"id":"2103.01210","title":"Quantifying the Benefit of Using Differentiable Learning over Tangent Kernels","created":"2021-03-01","cited_by_count":41,"area":null},{"id":"2207.04036","title":"Implicit Bias of Gradient Descent on Reparametrized Models: On Equivalence to Mirror Descent","created":"2022-07-08","cited_by_count":39,"area":null},{"id":"2306.04251","title":"Stochastic Collapse: How Gradient Noise Attracts SGD Dynamics Towards Simpler Subnetworks","created":"2023-06-07","cited_by_count":39,"area":null},{"id":"2307.11007","title":"Sharpness Minimization Algorithms Do Not Only Minimize Sharpness To Achieve Better Generalization","created":"2023-07-20","cited_by_count":39,"area":null},{"id":"2309.15111","title":"SGD Finds then Tunes Features in Two-Layer Neural Networks with near-Optimal Sample Complexity: A Case Study in the XOR problem","created":"2023-09-26","cited_by_count":39,"area":null},{"id":"2301.12832","title":"Deep networks for system identification: a Survey","created":"2023-01-30","cited_by_count":37,"area":null},{"id":"2302.08982","title":"(S)GD over Diagonal Linear Networks: Implicit Regularisation, Large Stepsizes and Edge of Stability","created":"2023-02-17","cited_by_count":36,"area":null},{"id":"2102.07346","title":"On the Theory of Implicit Deep Learning: Global Convergence with Implicit Layers","created":"2021-02-15","cited_by_count":35,"area":null},{"id":"2107.06608","title":"Continuous vs. Discrete Optimization of Deep Neural Networks","created":"2021-07-14","cited_by_count":34,"area":null},{"id":"2110.02732","title":"On Margin Maximization in Linear and ReLU Networks","created":"2021-10-06","cited_by_count":34,"area":null},{"id":"2203.02839","title":"Algorithmic Regularization in Model-free Overparametrized Asymmetric Matrix Factorization","created":"2022-03-06","cited_by_count":34,"area":null},{"id":"2306.03534","title":"Continual Learning in Linear Classification on Separable Data","created":"2023-06-06","cited_by_count":34,"area":null},{"id":"2106.04110","title":"A self consistent theory of Gaussian Processes captures feature learning effects in finite CNNs","created":"2021-06-08","cited_by_count":33,"area":null},{"id":"2202.01331","title":"Fast Convex Optimization for Two-Layer ReLU Networks: Equivalent Model Classes and Cone Decompositions","created":"2022-02-02","cited_by_count":31,"area":null},{"id":"2306.04815","title":"Catapults in SGD: spikes in the training loss and their impact on generalization through feature learning","created":"2023-06-07","cited_by_count":31,"area":null},{"id":"2108.05574","title":"Implicit Sparse Regularization: The Impact of Depth and Early Stopping","created":"2021-08-12","cited_by_count":30,"area":null},{"id":"2309.07418","title":"A Fast Optimization View: Reformulating Single Layer Attention in LLM Based on Tensor and SVM Trick, and Solving It in Matrix Multiplication Time","created":"2023-09-14","cited_by_count":30,"area":null},{"id":"2206.09841","title":"Label noise (stochastic) gradient descent implicitly solves the Lasso for quadratic parametrisation","created":"2022-06-20","cited_by_count":29,"area":null},{"id":"2604.21691","title":"There Will Be a Scientific Theory of Deep Learning","created":"2026-04-23","cited_by_count":29,"area":null},{"id":"2102.09385","title":"Convergence of stochastic gradient descent schemes for Lojasiewicz-landscapes","created":"2021-02-16","cited_by_count":28,"area":null},{"id":"2208.14673","title":"Incremental Learning in Diagonal Linear Networks","created":"2022-08-31","cited_by_count":28,"area":null},{"id":"2105.10585","title":"Properties of the After Kernel","created":"2021-05-21","cited_by_count":27,"area":null},{"id":"2201.11729","title":"Implicit Regularization in Hierarchical Tensor Factorization and Deep Convolutional Neural Networks","created":"2022-01-27","cited_by_count":27,"area":null},{"id":"2309.00079","title":"On the Implicit Bias of Adam","created":"2023-08-31","cited_by_count":27,"area":null},{"id":"2206.04613","title":"Explicit Regularization in Overparametrized Models via Noise Injection","created":"2022-06-09","cited_by_count":26,"area":null},{"id":"2307.04204","title":"Trajectory Alignment: Understanding the Edge of Stability Phenomenon via Bifurcation Theory","created":"2023-07-09","cited_by_count":26,"area":null},{"id":"2210.03820","title":"The Asymmetric Maximum Margin Bias of Quasi-Homogeneous Neural Networks","created":"2022-10-07","cited_by_count":25,"area":null},{"id":"2110.02914","title":"Foolish Crowds Support Benign Overfitting","created":"2021-10-06","cited_by_count":24,"area":null},{"id":"2112.11027","title":"More is Less: Inducing Sparsity via Overparameterization","created":"2021-12-21","cited_by_count":24,"area":null},{"id":"2203.00573","title":"Contrasting random and learned features in deep Bayesian linear regression","created":"2022-03-01","cited_by_count":24,"area":null},{"id":"2206.12314","title":"Learning sparse features can lead to overfitting in neural networks","created":"2022-06-24","cited_by_count":23,"area":null},{"id":"2504.12532","title":"Generalization through variance: how noise shapes inductive biases in diffusion models","created":"2025-04-16","cited_by_count":23,"area":"diffusion"},{"id":"2211.07844","title":"Characterizing the Spectrum of the NTK via a Power Series Expansion","created":"2022-11-15","cited_by_count":22,"area":null},{"id":"2205.09072","title":"On the Effective Number of Linear Regions in Shallow Univariate ReLU Networks: Convergence Guarantees and Implicit Bias","created":"2022-05-18","cited_by_count":21,"area":null},{"id":"2206.10451","title":"Winning the Lottery Ahead of Time: Efficient Early Network Pruning","created":"2022-06-21","cited_by_count":21,"area":null},{"id":"2305.13064","title":"Gradient Descent Monotonically Decreases the Sharpness of Gradient Flow Solutions in Scalar Networks and Beyond","created":"2023-05-22","cited_by_count":21,"area":null},{"id":"2106.06573","title":"Understanding Deflation Process in Over-parametrized Tensor Decomposition","created":"2021-06-11","cited_by_count":20,"area":null},{"id":"2211.16980","title":"Infinite-width limit of deep linear neural networks","created":"2022-11-29","cited_by_count":20,"area":null},{"id":"2306.16361","title":"Beyond NTK with Vanilla Gradient Descent: A Mean-Field Analysis of Neural Networks with Polynomial Width, Samples, and Time","created":"2023-06-28","cited_by_count":20,"area":null},{"id":"2308.13431","title":"Six Lectures on Linearized Neural Networks","created":"2023-08-25","cited_by_count":20,"area":null},{"id":"2405.16391","title":"When does compositional structure yield compositional generalization? A kernel theory","created":"2024-05-26","cited_by_count":20,"area":null},{"id":"2410.23213","title":"ELMGS: Enhancing memory and computation scaLability through coMpression for 3D Gaussian Splatting","created":"2024-10-30","cited_by_count":20,"area":null},{"id":"2101.02533","title":"Towards Understanding Learning in Neural Networks with Linear Teachers","created":"2021-01-07","cited_by_count":19,"area":null},{"id":"2305.06986","title":"Provable Guarantees for Nonlinear Feature Learning in Three-Layer Neural Networks","created":"2023-05-11","cited_by_count":19,"area":null},{"id":"2403.08699","title":"Implicit Regularization of Gradient Flow on One-Layer Softmax Attention","created":"2024-03-13","cited_by_count":19,"area":null},{"id":"2108.11489","title":"The Interplay Between Implicit Bias and Benign Overfitting in Two-Layer Linear Networks","created":"2021-08-25","cited_by_count":18,"area":null},{"id":"2111.05841","title":"Physics-enhanced deep surrogates for partial differential equations","created":"2021-11-10","cited_by_count":18,"area":null},{"id":"2209.07263","title":"Robustness in deep learning: The good (width), the bad (depth), and the ugly (initialization)","created":"2022-09-15","cited_by_count":18,"area":null},{"id":"2305.18512","title":"A Rainbow in Deep Network Black Boxes","created":"2023-05-29","cited_by_count":18,"area":null},{"id":"2402.06323","title":"How Uniform Random Weights Induce Non-uniform Bias: Typical Interpolating Neural Networks Generalize with Narrow Teachers","created":"2024-02-09","cited_by_count":18,"area":null},{"id":"2405.17580","title":"Mixed Dynamics In Linear Networks: Unifying the Lazy and Active Regimes","created":"2024-05-27","cited_by_count":18,"area":null},{"id":"2410.22069","title":"Flavors of Margin: Implicit Bias of Steepest Descent in Homogeneous Neural Networks","created":"2024-10-29","cited_by_count":18,"area":null},{"id":"2101.01152","title":"Provable Generalization of SGD-trained Neural Networks of Any Width in the Presence of Adversarial Label Noise","created":"2021-01-04","cited_by_count":17,"area":null},{"id":"2107.01641","title":"A Theoretical Analysis of Fine-tuning with Linear Teachers","created":"2021-07-04","cited_by_count":17,"area":null},{"id":"2402.05787","title":"How do Transformers perform In-Context Autoregressive Learning?","created":"2024-02-08","cited_by_count":17,"area":null},{"id":"2507.12224","title":"Optimizers Qualitatively Alter Solutions And We Should Leverage This","created":"2025-07-16","cited_by_count":17,"area":null}],"next_cursor":"100"}