{"total":303,"rows":[{"id":"2105.02375","title":"A Geometric Analysis of Neural Collapse with Unconstrained Features","created":"2021-05-06","cited_by_count":227,"area":null},{"id":"2206.07758","title":"Reconstructing Training Data from Trained Neural Networks","created":"2022-06-15","cited_by_count":133,"area":null},{"id":"2106.11230","title":"Can contrastive learning avoid shortcut solutions?","created":"2021-06-21","cited_by_count":114,"area":null},{"id":"2309.02390","title":"Explaining grokking through circuit efficiency","created":"2023-09-05","cited_by_count":111,"area":null},{"id":"2106.09524","title":"Implicit Bias of SGD for Diagonal Linear Networks: a Provable Benefit of Stochasticity","created":"2021-06-17","cited_by_count":110,"area":null},{"id":"2110.06914","title":"What Happens after SGD Reaches Zero Loss? --A Mathematical Framework","created":"2021-10-13","cited_by_count":107,"area":null},{"id":"2207.14484","title":"Adaptive Gradient Methods at the Edge of Stability","created":"2022-07-29","cited_by_count":101,"area":null},{"id":"2206.04041","title":"Neural Collapse: A Review on Modelling Principles and Generalization","created":"2022-06-08","cited_by_count":95,"area":null},{"id":"2206.04817","title":"The Slingshot Mechanism: An Empirical Study of Adaptive Optimizers and the Grokking Phenomenon","created":"2022-06-10","cited_by_count":94,"area":null},{"id":"2210.09221","title":"Vision Transformers provably learn spatial structure","created":"2022-10-13","cited_by_count":93,"area":null},{"id":"2210.15651","title":"Learning Single-Index Models with Shallow Neural Networks","created":"2022-10-27","cited_by_count":92,"area":null},{"id":"2202.06526","title":"Benign Overfitting in Two-layer Convolutional Neural Networks","created":"2022-02-14","cited_by_count":91,"area":null},{"id":"2110.02796","title":"An Unconstrained Layer-Peeled Perspective on Neural Collapse","created":"2021-10-06","cited_by_count":87,"area":null},{"id":"2206.07085","title":"Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction","created":"2022-06-14","cited_by_count":86,"area":null},{"id":"2208.12591","title":"On the Implicit Bias in Deep-Learning Algorithms","created":"2022-08-26","cited_by_count":86,"area":null},{"id":"2106.15933","title":"Saddle-to-Saddle Dynamics in Deep Linear Networks: Small Initialization Training, Symmetry, and Sparsity","created":"2021-06-30","cited_by_count":85,"area":null},{"id":"2210.14199","title":"Same Pre-training Loss, Better Downstream: Implicit Bias Matters for Language Models","created":"2022-10-25","cited_by_count":84,"area":null},{"id":"2110.13905","title":"Gradient Descent on Two-layer Nets: Margin Maximization and Simplicity Bias","created":"2021-10-26","cited_by_count":83,"area":null},{"id":"2208.05512","title":"Imbalance Trouble: Revisiting Neural-Collapse Geometry","created":"2022-08-10","cited_by_count":80,"area":null},{"id":"2311.18817","title":"Dichotomy of Early and Late Phase Implicit Biases Can Provably Induce Grokking","created":"2023-11-30","cited_by_count":80,"area":null},{"id":"2206.00939","title":"Gradient flow dynamics of shallow ReLU networks for square loss and orthogonal inputs","created":"2022-06-02","cited_by_count":76,"area":null},{"id":"2308.16898","title":"Transformers as Support Vector Machines","created":"2023-08-31","cited_by_count":74,"area":null},{"id":"2102.09769","title":"On the Implicit Bias of Initialization Shape: Beyond Infinitesimal Mirror Descent","created":"2021-02-19","cited_by_count":70,"area":null},{"id":"2210.05337","title":"SGD with Large Step Sizes Learns Sparse Features","created":"2022-10-11","cited_by_count":68,"area":null},{"id":"2105.11144","title":"Improved OOD Generalization via Adversarial Training and Pre-training","created":"2021-05-24","cited_by_count":66,"area":null},{"id":"2306.13596","title":"Max-Margin Token Selection in Attention Mechanism","created":"2023-06-23","cited_by_count":58,"area":null},{"id":"2404.04454","title":"Implicit Bias of AdamW: $\\ell_\\infty$ Norm Constrained Optimization","created":"2024-04-05","cited_by_count":58,"area":null},{"id":"2210.05775","title":"C-Mixup: Improving Generalization in Regression","created":"2022-10-11","cited_by_count":57,"area":null},{"id":"2304.00488","title":"Saddle-to-Saddle Dynamics in Diagonal Linear Networks","created":"2023-04-02","cited_by_count":57,"area":null},{"id":"2201.12760","title":"Implicit Regularization Towards Rank Minimization in ReLU Networks","created":"2022-01-30","cited_by_count":53,"area":null},{"id":"2211.08422","title":"Mechanistic Mode Connectivity","created":"2022-11-15","cited_by_count":50,"area":null},{"id":"2211.11567","title":"Neural networks trained with SGD learn distributions of increasing complexity","created":"2022-11-21","cited_by_count":50,"area":null},{"id":"2104.13628","title":"Risk Bounds for Over-parameterized Maximum Margin Classification on Sub-Gaussian Mixtures","created":"2021-04-28","cited_by_count":49,"area":null},{"id":"2210.07082","title":"Implicit Bias in Leaky ReLU Networks Trained on High-Dimensional Data","created":"2022-10-13","cited_by_count":46,"area":null},{"id":"2501.04697","title":"Grokking at the Edge of Numerical Stability","created":"2025-01-08","cited_by_count":46,"area":null},{"id":"2303.15725","title":"Solving Regularized Exp, Cosh and Sinh Regression Problems","created":"2023-03-28","cited_by_count":44,"area":null},{"id":"2102.04877","title":"Noisy Recurrent Neural Networks","created":"2021-02-09","cited_by_count":42,"area":null},{"id":"2301.11500","title":"Understanding Incremental Learning of Gradient Descent: A Fine-grained Analysis of Matrix Sensing","created":"2023-01-27","cited_by_count":42,"area":null},{"id":"2305.11788","title":"Implicit Bias of Gradient Descent for Logistic Regression at the Edge of Stability","created":"2023-05-19","cited_by_count":42,"area":null},{"id":"2406.06158","title":"Get rich quick: exact solutions reveal how unbalanced initializations promote rapid feature learning","created":"2024-06-10","cited_by_count":42,"area":null},{"id":"2502.04664","title":"Implicit Bias of Spectral Descent and Muon on Multiclass Separable Data","created":"2025-02-07","cited_by_count":41,"area":null},{"id":"2207.05931","title":"Towards understanding how momentum improves generalization in deep learning","created":"2022-07-13","cited_by_count":40,"area":null},{"id":"2103.15209","title":"Understanding the role of importance weighting for deep learning","created":"2021-03-28","cited_by_count":39,"area":null},{"id":"2207.04036","title":"Implicit Bias of Gradient Descent on Reparametrized Models: On Equivalence to Mirror Descent","created":"2022-07-08","cited_by_count":39,"area":null},{"id":"2309.15111","title":"SGD Finds then Tunes Features in Two-Layer Neural Networks with near-Optimal Sample Complexity: A Case Study in the XOR problem","created":"2023-09-26","cited_by_count":39,"area":null},{"id":"2205.06226","title":"The Mechanism of Prediction Head in Non-contrastive Self-supervised Learning","created":"2022-05-12","cited_by_count":38,"area":null},{"id":"2302.08982","title":"(S)GD over Diagonal Linear Networks: Implicit Regularisation, Large Stepsizes and Edge of Stability","created":"2023-02-17","cited_by_count":36,"area":null},{"id":"2406.10650","title":"The Implicit Bias of Adam on Separable Data","created":"2024-06-15","cited_by_count":36,"area":null},{"id":"2407.12332","title":"Why Do You Grok? A Theoretical Analysis of Grokking Modular Addition","created":"2024-07-17","cited_by_count":36,"area":null},{"id":"2106.05932","title":"Early-stopped neural networks are consistent","created":"2021-06-10","cited_by_count":34,"area":null},{"id":"2107.06608","title":"Continuous vs. Discrete Optimization of Deep Neural Networks","created":"2021-07-14","cited_by_count":34,"area":null},{"id":"2110.02732","title":"On Margin Maximization in Linear and ReLU Networks","created":"2021-10-06","cited_by_count":34,"area":null},{"id":"2107.00595","title":"Fast Margin Maximization via Dual Acceleration","created":"2021-07-01","cited_by_count":33,"area":null},{"id":"2201.11968","title":"Training invariances and the low-rank phenomenon: beyond linear networks","created":"2022-01-28","cited_by_count":33,"area":null},{"id":"2311.07568","title":"Feature emergence via margin maximization: case studies in algebraic tasks","created":"2023-11-13","cited_by_count":33,"area":null},{"id":"2402.05738","title":"Implicit Bias and Fast Convergence Rates for Self-attention","created":"2024-02-08","cited_by_count":31,"area":null},{"id":"2206.09841","title":"Label noise (stochastic) gradient descent implicitly solves the Lasso for quadratic parametrisation","created":"2022-06-20","cited_by_count":29,"area":null},{"id":"2211.05641","title":"Regression as Classification: Influence of Task Formulation on Neural Network Features","created":"2022-11-10","cited_by_count":29,"area":null},{"id":"2303.01462","title":"Benign Overfitting in Linear Classifiers and Leaky ReLU Networks from KKT Conditions for Margin Maximization","created":"2023-03-02","cited_by_count":29,"area":null},{"id":"2306.02295","title":"A Mathematical Abstraction for Balancing the Trade-off Between Creativity and Reality in Large Language Models","created":"2023-06-04","cited_by_count":29,"area":null},{"id":"2604.21691","title":"There Will Be a Scientific Theory of Deep Learning","created":"2026-04-23","cited_by_count":29,"area":null},{"id":"2307.12851","title":"Early Neuron Alignment in Two-layer ReLU Networks with Small Initialization","created":"2023-07-24","cited_by_count":28,"area":null},{"id":"2105.10585","title":"Properties of the After Kernel","created":"2021-05-21","cited_by_count":27,"area":null},{"id":"2201.11729","title":"Implicit Regularization in Hierarchical Tensor Factorization and Deep Convolutional Neural Networks","created":"2022-01-27","cited_by_count":27,"area":null},{"id":"2202.04347","title":"Gradient Methods Provably Converge to Non-Robust Networks","created":"2022-02-09","cited_by_count":27,"area":null},{"id":"2309.00079","title":"On the Implicit Bias of Adam","created":"2023-08-31","cited_by_count":27,"area":null},{"id":"2310.20703","title":"Vanishing Gradients in Reinforcement Finetuning of Language Models","created":"2023-10-31","cited_by_count":27,"area":null},{"id":"2205.12808","title":"Mirror Descent Maximizes Generalized Margin and Can Be Implemented Efficiently","created":"2022-05-25","cited_by_count":26,"area":null},{"id":"2206.04613","title":"Explicit Regularization in Overparametrized Models via Noise Injection","created":"2022-06-09","cited_by_count":26,"area":null},{"id":"2307.01827","title":"Deconstructing Data Reconstruction: Multiclass, Weight Decay and General Losses","created":"2023-07-04","cited_by_count":26,"area":null},{"id":"2309.04332","title":"Graph Neural Networks Use Graphs When They Shouldn't","created":"2023-09-08","cited_by_count":26,"area":null},{"id":"2210.03820","title":"The Asymmetric Maximum Margin Bias of Quasi-Homogeneous Neural Networks","created":"2022-10-07","cited_by_count":25,"area":null},{"id":"2408.15417","title":"Implicit Geometry of Next-token Prediction: From Language Sparsity Patterns to Model Representations","created":"2024-08-27","cited_by_count":24,"area":null},{"id":"2303.01456","title":"The Double-Edged Sword of Implicit Bias: Generalization vs. Robustness in ReLU Networks","created":"2023-03-02","cited_by_count":23,"area":null},{"id":"2303.07608","title":"On the Implicit Geometry of Cross-Entropy Parameterizations for Label-Imbalanced Data","created":"2023-03-14","cited_by_count":23,"area":null},{"id":"2602.07852","title":"Emergent Misalignment is Easy, Narrow Misalignment is Hard","created":"2026-02-08","cited_by_count":23,"area":null},{"id":"2110.03891","title":"Does Momentum Change the Implicit Regularization on Separable Data?","created":"2021-10-08","cited_by_count":22,"area":null},{"id":"2203.03597","title":"Fast Rates for Noisy Interpolation Require Rethinking the Effects of Inductive Bias","created":"2022-03-07","cited_by_count":22,"area":null},{"id":"2205.09072","title":"On the Effective Number of Linear Regions in Shallow Univariate ReLU Networks: Convergence Guarantees and Implicit Bias","created":"2022-05-18","cited_by_count":21,"area":null},{"id":"2302.09235","title":"Generalization and Stability of Interpolating Neural Networks with Minimal Width","created":"2023-02-18","cited_by_count":21,"area":null},{"id":"2106.06573","title":"Understanding Deflation Process in Over-parametrized Tensor Decomposition","created":"2021-06-11","cited_by_count":20,"area":null},{"id":"2308.13431","title":"Six Lectures on Linearized Neural Networks","created":"2023-08-25","cited_by_count":20,"area":null},{"id":"2309.01213","title":"Implicit regularization of deep residual networks towards neural ODEs","created":"2023-09-03","cited_by_count":20,"area":null},{"id":"2405.16391","title":"When does compositional structure yield compositional generalization? A kernel theory","created":"2024-05-26","cited_by_count":20,"area":null},{"id":"2101.02533","title":"Towards Understanding Learning in Neural Networks with Linear Teachers","created":"2021-01-07","cited_by_count":19,"area":null},{"id":"2110.06084","title":"Implicit Bias of Linear Equivariant Networks","created":"2021-10-12","cited_by_count":19,"area":null},{"id":"2112.12986","title":"Is Importance Weighting Incompatible with Interpolating Classifiers?","created":"2021-12-24","cited_by_count":19,"area":null},{"id":"2303.01353","title":"Penalising the biases in norm regularisation enforces sparsity","created":"2023-03-02","cited_by_count":19,"area":null},{"id":"2403.08699","title":"Implicit Regularization of Gradient Flow on One-Layer Softmax Attention","created":"2024-03-13","cited_by_count":19,"area":null},{"id":"2102.12238","title":"Inductive Bias of Multi-Channel Linear Convolutional Networks with Bounded Weight Norm","created":"2021-02-24","cited_by_count":18,"area":null},{"id":"2310.18935","title":"Implicit Bias of Gradient Descent for Two-layer ReLU and Leaky ReLU Networks on Nearly-orthogonal Data","created":"2023-10-29","cited_by_count":18,"area":null},{"id":"2402.06323","title":"How Uniform Random Weights Induce Non-uniform Bias: Typical Interpolating Neural Networks Generalize with Narrow Teachers","created":"2024-02-09","cited_by_count":18,"area":null},{"id":"2410.22069","title":"Flavors of Margin: Implicit Bias of Steepest Descent in Homogeneous Neural Networks","created":"2024-10-29","cited_by_count":18,"area":null},{"id":"2101.01152","title":"Provable Generalization of SGD-trained Neural Networks of Any Width in the Presence of Adversarial Label Noise","created":"2021-01-04","cited_by_count":17,"area":null},{"id":"2107.01641","title":"A Theoretical Analysis of Fine-tuning with Linear Teachers","created":"2021-07-04","cited_by_count":17,"area":null},{"id":"2111.06063","title":"On the Equivalence between Neural Network and Support Vector Machine","created":"2021-11-11","cited_by_count":17,"area":null},{"id":"2202.06915","title":"Stochastic linear optimization never overfits with quadratically-bounded losses on general data","created":"2022-02-14","cited_by_count":17,"area":null},{"id":"2205.13647","title":"Learning to Reason with Neural Networks: Generalization, Unseen Data and Boolean Measures","created":"2022-05-26","cited_by_count":17,"area":null},{"id":"2406.08654","title":"Large Stepsize Gradient Descent for Non-Homogeneous Two-Layer Networks: Margin Improvement and Fast Optimization","created":"2024-06-12","cited_by_count":17,"area":null},{"id":"2305.15141","title":"From Tempered to Benign Overfitting in ReLU Neural Networks","created":"2023-05-24","cited_by_count":16,"area":null}],"next_cursor":"100"}