{"total":143,"rows":[{"id":"2306.09927","title":"Trained Transformers Learn Linear Models In-Context","created":"2023-06-16","cited_by_count":337,"area":null},{"id":"2103.10427","title":"The Low-Rank Simplicity Bias in Deep Networks","created":"2021-03-18","cited_by_count":154,"area":null},{"id":"2206.07758","title":"Reconstructing Training Data from Trained Neural Networks","created":"2022-06-15","cited_by_count":133,"area":null},{"id":"2205.09745","title":"Understanding Gradient Descent on Edge of Stability in Deep Learning","created":"2022-05-19","cited_by_count":128,"area":null},{"id":"2110.06914","title":"What Happens after SGD Reaches Zero Loss? --A Mathematical Framework","created":"2021-10-13","cited_by_count":107,"area":null},{"id":"2202.14026","title":"Robust Training under Label Noise by Over-parameterization","created":"2022-02-28","cited_by_count":93,"area":null},{"id":"2206.07085","title":"Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction","created":"2022-06-14","cited_by_count":86,"area":null},{"id":"2208.12591","title":"On the Implicit Bias in Deep-Learning Algorithms","created":"2022-08-26","cited_by_count":86,"area":null},{"id":"2106.15933","title":"Saddle-to-Saddle Dynamics in Deep Linear Networks: Small Initialization Training, Symmetry, and Sparsity","created":"2021-06-30","cited_by_count":85,"area":null},{"id":"2210.14199","title":"Same Pre-training Loss, Better Downstream: Implicit Bias Matters for Language Models","created":"2022-10-25","cited_by_count":84,"area":null},{"id":"2110.13905","title":"Gradient Descent on Two-layer Nets: Margin Maximization and Simplicity Bias","created":"2021-10-26","cited_by_count":83,"area":null},{"id":"2311.18817","title":"Dichotomy of Early and Late Phase Implicit Biases Can Provably Induce Grokking","created":"2023-11-30","cited_by_count":80,"area":null},{"id":"2106.15013","title":"Small random initialization is akin to spectral learning: Optimization and generalization guarantees for overparameterized low-rank matrix reconstruction","created":"2021-06-28","cited_by_count":78,"area":null},{"id":"2206.00939","title":"Gradient flow dynamics of shallow ReLU networks for square loss and orthogonal inputs","created":"2022-06-02","cited_by_count":76,"area":null},{"id":"2102.09769","title":"On the Implicit Bias of Initialization Shape: Beyond Infinitesimal Mirror Descent","created":"2021-02-19","cited_by_count":70,"area":null},{"id":"2303.00055","title":"Learning time-scales in two-layers neural networks","created":"2023-02-28","cited_by_count":57,"area":null},{"id":"2304.00488","title":"Saddle-to-Saddle Dynamics in Diagonal Linear Networks","created":"2023-04-02","cited_by_count":57,"area":null},{"id":"2310.19793","title":"On Learning Gaussian Multi-index Models with Gradient Flow","created":"2023-10-30","cited_by_count":57,"area":null},{"id":"2201.12760","title":"Implicit Regularization Towards Rank Minimization in ReLU Networks","created":"2022-01-30","cited_by_count":53,"area":null},{"id":"2306.07042","title":"Transformers learn through gradual rank increase","created":"2023-06-12","cited_by_count":52,"area":null},{"id":"2409.14623","title":"From Lazy to Rich: Exact Learning Dynamics in Deep Linear Networks","created":"2024-09-22","cited_by_count":49,"area":null},{"id":"2210.07082","title":"Implicit Bias in Leaky ReLU Networks Trained on High-Dimensional Data","created":"2022-10-13","cited_by_count":46,"area":null},{"id":"2102.09972","title":"Implicit Regularization in Tensor Factorization","created":"2021-02-19","cited_by_count":44,"area":null},{"id":"2301.11500","title":"Understanding Incremental Learning of Gradient Descent: A Fine-grained Analysis of Matrix Sensing","created":"2023-01-27","cited_by_count":42,"area":null},{"id":"2406.06158","title":"Get rich quick: exact solutions reveal how unbalanced initializations promote rapid feature learning","created":"2024-06-10","cited_by_count":42,"area":null},{"id":"2410.02984","title":"Differentiation and Specialization of Attention Heads via the Refined Local Learning Coefficient","created":"2024-10-03","cited_by_count":40,"area":null},{"id":"2207.04036","title":"Implicit Bias of Gradient Descent on Reparametrized Models: On Equivalence to Mirror Descent","created":"2022-07-08","cited_by_count":39,"area":null},{"id":"2209.15055","title":"Implicit Bias of Large Depth Networks: a Notion of Rank for Nonlinear Functions","created":"2022-09-29","cited_by_count":38,"area":null},{"id":"2110.02732","title":"On Margin Maximization in Linear and ReLU Networks","created":"2021-10-06","cited_by_count":34,"area":null},{"id":"2402.05738","title":"Implicit Bias and Fast Convergence Rates for Self-attention","created":"2024-02-08","cited_by_count":31,"area":null},{"id":"2408.11804","title":"Approaching Deep Learning through the Spectral Dynamics of Weights","created":"2024-08-21","cited_by_count":31,"area":null},{"id":"2108.05574","title":"Implicit Sparse Regularization: The Impact of Depth and Early Stopping","created":"2021-08-12","cited_by_count":30,"area":null},{"id":"2305.16038","title":"Implicit bias of SGD in $L_{2}$-regularized linear DNNs: One-way jumps from high to low rank","created":"2023-05-25","cited_by_count":29,"area":null},{"id":"2604.21691","title":"There Will Be a Scientific Theory of Deep Learning","created":"2026-04-23","cited_by_count":29,"area":null},{"id":"2110.12661","title":"ZerO Initialization: Initializing Neural Networks with only Zeros and Ones","created":"2021-10-25","cited_by_count":28,"area":null},{"id":"2208.14673","title":"Incremental Learning in Diagonal Linear Networks","created":"2022-08-31","cited_by_count":28,"area":null},{"id":"2307.12851","title":"Early Neuron Alignment in Two-layer ReLU Networks with Small Initialization","created":"2023-07-24","cited_by_count":28,"area":null},{"id":"2201.11729","title":"Implicit Regularization in Hierarchical Tensor Factorization and Deep Convolutional Neural Networks","created":"2022-01-27","cited_by_count":27,"area":null},{"id":"2410.23819","title":"Weight decay induces low-rank attention layers","created":"2024-10-31","cited_by_count":27,"area":null},{"id":"2302.00457","title":"Simplicity Bias in 1-Hidden Layer Neural Networks","created":"2023-02-01","cited_by_count":26,"area":null},{"id":"2406.04112","title":"Compressible Dynamics in Deep Overparameterized Low-Rank Learning & Adaptation","created":"2024-06-06","cited_by_count":26,"area":null},{"id":"2408.03733","title":"Bayes-optimal learning of an extensive-width neural network from quadratically many samples","created":"2024-08-07","cited_by_count":23,"area":null},{"id":"2402.13728","title":"Average gradient outer product as a mechanism for deep neural collapse","created":"2024-02-21","cited_by_count":22,"area":null},{"id":"2202.08788","title":"Global Convergence of Sub-gradient Method for Robust Matrix Recovery: Small Initialization, Noisy Measurements, and Over-parameterization","created":"2022-02-17","cited_by_count":21,"area":null},{"id":"2106.06573","title":"Understanding Deflation Process in Over-parametrized Tensor Decomposition","created":"2021-06-11","cited_by_count":20,"area":null},{"id":"2211.16980","title":"Infinite-width limit of deep linear neural networks","created":"2022-11-29","cited_by_count":20,"area":null},{"id":"2308.13431","title":"Six Lectures on Linearized Neural Networks","created":"2023-08-25","cited_by_count":20,"area":null},{"id":"2309.01213","title":"Implicit regularization of deep residual networks towards neural ODEs","created":"2023-09-03","cited_by_count":20,"area":null},{"id":"2303.01353","title":"Penalising the biases in norm regularisation enforces sparsity","created":"2023-03-02","cited_by_count":19,"area":null},{"id":"2403.08699","title":"Implicit Regularization of Gradient Flow on One-Layer Softmax Attention","created":"2024-03-13","cited_by_count":19,"area":null},{"id":"2102.12238","title":"Inductive Bias of Multi-Channel Linear Convolutional Networks with Bounded Weight Norm","created":"2021-02-24","cited_by_count":18,"area":null},{"id":"2310.01769","title":"How Over-Parameterization Slows Down Gradient Descent in Matrix Sensing: The Curses of Symmetry and Initialization","created":"2023-10-03","cited_by_count":18,"area":null},{"id":"2405.17580","title":"Mixed Dynamics In Linear Networks: Unifying the Lazy and Active Regimes","created":"2024-05-27","cited_by_count":18,"area":null},{"id":"2205.15809","title":"Feature Learning in $L_{2}$-regularized DNNs: Attraction/Repulsion and Sparsity","created":"2022-05-31","cited_by_count":17,"area":null},{"id":"2405.13456","title":"Deep linear networks for regression are implicitly regularized towards flat minima","created":"2024-05-22","cited_by_count":17,"area":null},{"id":"2306.06479","title":"Learning a Neuron by a Shallow ReLU Network: Dynamics and Implicit Bias for Correlated Inputs","created":"2023-06-10","cited_by_count":16,"area":null},{"id":"2508.12837","title":"Learning In-context n-grams with Transformers: Sub-n-grams Are Near-stationary Points","created":"2025-08-18","cited_by_count":16,"area":null},{"id":"2206.08756","title":"Tensor-on-Tensor Regression: Riemannian Optimization, Over-parameterization, Statistical-computational Gap, and Their Interplay","created":"2022-06-17","cited_by_count":15,"area":null},{"id":"2512.20607","title":"Saddle-to-Saddle Dynamics Explains A Simplicity Bias Across Neural Network Architectures","created":"2025-12-23","cited_by_count":15,"area":null},{"id":"2403.06925","title":"Transformers Learn Low Sensitivity Functions: Investigations and Implications","created":"2024-03-11","cited_by_count":14,"area":null},{"id":"2505.21289","title":"LoFT: Low-Rank Adaptation That Behaves Like Full Fine-Tuning","created":"2025-05-27","cited_by_count":14,"area":null},{"id":"2506.06489","title":"Alternating Gradient Flows: A Theory of Feature Learning in Two-layer Neural Networks","created":"2025-06-06","cited_by_count":14,"area":null},{"id":"2410.23169","title":"The Persistence of Neural Collapse Despite Low-Rank Bias","created":"2024-10-30","cited_by_count":13,"area":null},{"id":"2505.24022","title":"The Rich and the Simple: On the Implicit Bias of Adam and SGD","created":"2025-05-29","cited_by_count":13,"area":null},{"id":"2504.12883","title":"Mirror, Mirror of the Flow: How Does Regularization Shape Implicit Bias?","created":"2025-04-17","cited_by_count":12,"area":null},{"id":"2105.13831","title":"Implicit Regularization in Matrix Sensing via Mirror Descent","created":"2021-05-28","cited_by_count":11,"area":null},{"id":"2112.06868","title":"Variational autoencoders in the presence of low-dimensional data: landscape and implicit bias","created":"2021-12-13","cited_by_count":11,"area":null},{"id":"2204.05023","title":"Machine Learning and Deep Learning -- A review for Ecologists","created":"2022-04-11","cited_by_count":11,"area":null},{"id":"2306.11250","title":"InRank: Incremental Low-Rank Learning","created":"2023-06-20","cited_by_count":10,"area":null},{"id":"2602.03655","title":"Sequential Group Composition: A Window into the Mechanics of Deep Learning","created":"2026-02-03","cited_by_count":10,"area":null},{"id":"2603.00742","title":"To Use or not to Use Muon: How Simplicity Bias in Optimizers Matters","created":"2026-02-28","cited_by_count":10,"area":null},{"id":"2402.08010","title":"Which Frequencies do CNNs Need? Emergent Bottleneck Structure in Feature Learning","created":"2024-02-12","cited_by_count":9,"area":null},{"id":"2410.14581","title":"Optimizing Attention with Mirror Descent: Generalized Max-Margin Token Selection","created":"2024-10-18","cited_by_count":9,"area":null},{"id":"2502.21009","title":"Position: Solve Layerwise Linear Models First to Understand Neural Dynamical Phenomena (Neural Collapse, Emergence, Lazy/Rich Regime, and Grokking)","created":"2025-02-28","cited_by_count":9,"area":null},{"id":"2503.19859","title":"An Overview of Low-Rank Structures in the Training and Adaptation of Large Models","created":"2025-03-25","cited_by_count":9,"area":null},{"id":"2209.09315","title":"Deep Linear Networks can Benignly Overfit when Shallow Ones Do","created":"2022-09-19","cited_by_count":8,"area":null},{"id":"2306.11680","title":"The Implicit Bias of Batch Normalization in Linear Models and Two-layer Linear Convolutional Neural Networks","created":"2023-06-20","cited_by_count":8,"area":null},{"id":"2502.09863","title":"Closed-Form Training Dynamics Reveal Learned Features and Linear Structure in Word2Vec-like Models","created":"2025-02-14","cited_by_count":8,"area":null},{"id":"2210.14064","title":"Learning Low Dimensional State Spaces with Overparameterized Recurrent Neural Nets","created":"2022-10-25","cited_by_count":7,"area":null},{"id":"2403.08121","title":"Early Directional Convergence in Deep Homogeneous Neural Networks for Small Initializations","created":"2024-03-12","cited_by_count":7,"area":null},{"id":"2509.25351","title":"Gradient Descent with Large Step Sizes: Chaos and Fractal Convergence Region","created":"2025-09-29","cited_by_count":7,"area":null},{"id":"2306.13239","title":"The Inductive Bias of Flatness Regularization for Deep Matrix Factorization","created":"2023-06-22","cited_by_count":6,"area":null},{"id":"2407.05664","title":"How DNNs break the Curse of Dimensionality: Compositionality and Symmetry Learning","created":"2024-07-08","cited_by_count":6,"area":null},{"id":"2305.09145","title":"Deep ReLU Networks Have Surprisingly Simple Polytopes","created":"2023-05-16","cited_by_count":5,"area":null},{"id":"2307.01177","title":"Neural Hilbert Ladders: Multi-Layer Neural Networks in Function Space","created":"2023-07-03","cited_by_count":5,"area":null},{"id":"2310.14423","title":"A Quadratic Synchronization Rule for Distributed Deep Learning","created":"2023-10-22","cited_by_count":5,"area":null},{"id":"2210.00346","title":"Behind the Scenes of Gradient Descent: A Trajectory Analysis via Basis Function Decomposition","created":"2022-10-01","cited_by_count":4,"area":null},{"id":"2212.09396","title":"Rank-1 Matrix Completion with Gradient Descent and Small Random Initialization","created":"2022-12-19","cited_by_count":4,"area":null},{"id":"2302.00257","title":"Implicit Regularization Leads to Benign Overfitting for Sparse Linear Regression","created":"2023-02-01","cited_by_count":4,"area":null},{"id":"2505.21722","title":"Saddle-To-Saddle Dynamics in Deep ReLU Networks: Low-Rank Bias in the First Saddle Escape","created":"2025-05-27","cited_by_count":4,"area":null},{"id":"2509.18766","title":"Diagonal Linear Networks and the Lasso Regularization Path","created":"2025-09-23","cited_by_count":4,"area":null},{"id":"2604.09258","title":"Nexus: Same Pretraining Loss, Better Downstream Generalization via Common Minima","created":"2026-04-10","cited_by_count":4,"area":null},{"id":"2207.07612","title":"Blessing of Nonconvexity in Deep Linear Models: Depth Flattens the Optimization Landscape Around the True Solution","created":"2022-07-15","cited_by_count":3,"area":null},{"id":"2207.08942","title":"Implicit Regularization with Polynomial Growth in Deep Tensor Factorization","created":"2022-07-18","cited_by_count":3,"area":null},{"id":"2311.05061","title":"Efficient Compression of Overparameterized Deep Models through Low-Dimensional Learning Dynamics","created":"2023-11-08","cited_by_count":3,"area":null},{"id":"2405.13721","title":"Connectivity Shapes Implicit Regularization in Matrix Factorization Models for Matrix Completion","created":"2024-05-22","cited_by_count":3,"area":null},{"id":"2409.00894","title":"Improving Adaptivity via Over-Parameterization in Sequence Models","created":"2024-09-02","cited_by_count":3,"area":null},{"id":"2410.18858","title":"Bilinear Sequence Regression: A Model for Learning from Long Sequences of High-dimensional Tokens","created":"2024-10-24","cited_by_count":3,"area":null},{"id":"2510.01175","title":"On the Benefits of Weight Normalization for Overparameterized Matrix Sensing","created":"2025-10-01","cited_by_count":3,"area":null},{"id":"2604.21905","title":"Low-Rank Adaptation Redux for Large Models","created":"2026-04-23","cited_by_count":3,"area":null}],"next_cursor":"100"}