{"total":408,"rows":[{"id":"2209.04836","title":"Git Re-Basin: Merging Models modulo Permutation Symmetries","created":"2022-09-11","cited_by_count":471,"area":null},{"id":"2312.10794","title":"A mathematical perspective on Transformers","created":"2023-12-17","cited_by_count":164,"area":null},{"id":"2206.15144","title":"Neural Networks can Learn Representations with Gradient Descent","created":"2022-06-30","cited_by_count":161,"area":null},{"id":"2106.02073","title":"Neural Collapse Under MSE Loss: Proximity to and Dynamics on the Central Path","created":"2021-06-03","cited_by_count":158,"area":null},{"id":"2205.01445","title":"High-dimensional Asymptotics of Feature Learning: How One Gradient Step Improves the Representation","created":"2022-05-03","cited_by_count":150,"area":null},{"id":"2302.11055","title":"SGD learning on neural networks: leap complexity and saddle-to-saddle dynamics","created":"2023-02-21","cited_by_count":143,"area":null},{"id":"2105.12221","title":"Geometry of the Loss Landscape in Overparameterized Neural Networks: Symmetries and Invariances","created":"2021-05-25","cited_by_count":127,"area":null},{"id":"2106.09524","title":"Implicit Bias of SGD for Diagonal Linear Networks: a Provable Benefit of Stochasticity","created":"2021-06-17","cited_by_count":110,"area":null},{"id":"2205.09653","title":"Self-Consistent Dynamical Field Theory of Kernel Evolution in Wide Neural Networks","created":"2022-05-19","cited_by_count":105,"area":null},{"id":"2305.16380","title":"Scan and Snap: Understanding Training Dynamics and Token Composition in 1-layer Transformer","created":"2023-05-25","cited_by_count":102,"area":null},{"id":"2206.04041","title":"Neural Collapse: A Review on Modelling Principles and Generalization","created":"2022-06-08","cited_by_count":95,"area":null},{"id":"2106.15933","title":"Saddle-to-Saddle Dynamics in Deep Linear Networks: Small Initialization Training, Symmetry, and Sparsity","created":"2021-06-30","cited_by_count":85,"area":null},{"id":"2106.15013","title":"Small random initialization is akin to spectral learning: Optimization and generalization guarantees for overparameterized low-rank matrix reconstruction","created":"2021-06-28","cited_by_count":78,"area":null},{"id":"2505.01618","title":"Don't be lazy: CompleteP enables compute-efficient deep transformers","created":"2025-05-02","cited_by_count":78,"area":null},{"id":"2206.00939","title":"Gradient flow dynamics of shallow ReLU networks for square loss and orthogonal inputs","created":"2022-06-02","cited_by_count":76,"area":null},{"id":"2206.04030","title":"High-dimensional limit theorems for SGD: Effective dynamics and critical scaling","created":"2022-06-08","cited_by_count":70,"area":null},{"id":"2102.11742","title":"Classifying high-dimensional Gaussian mixtures: Where kernel methods fail and neural networks succeed","created":"2021-02-23","cited_by_count":69,"area":null},{"id":"2201.07395","title":"Overview frequency principle/spectral bias in deep learning","created":"2022-01-19","cited_by_count":69,"area":null},{"id":"2305.18270","title":"How Two-Layer Neural Networks Learn, One (Giant) Step at a Time","created":"2023-05-29","cited_by_count":68,"area":null},{"id":"2309.16620","title":"Depthwise Hyperparameter Transfer in Residual Networks: Dynamics and Scaling Limit","created":"2023-09-28","cited_by_count":66,"area":null},{"id":"2209.14863","title":"Neural Networks Efficiently Learn Low-Dimensional Representations with SGD","created":"2022-09-29","cited_by_count":63,"area":null},{"id":"2303.00055","title":"Learning time-scales in two-layers neural networks","created":"2023-02-28","cited_by_count":57,"area":null},{"id":"2201.10469","title":"Convex Analysis of the Mean Field Langevin Dynamics","created":"2022-01-25","cited_by_count":55,"area":null},{"id":"2102.12781","title":"Do Input Gradients Highlight Discriminative Features?","created":"2021-02-25","cited_by_count":52,"area":null},{"id":"2310.00535","title":"JoMA: Demystifying Multilayer Transformers via JOint Dynamics of MLP and Attention","created":"2023-10-01","cited_by_count":52,"area":null},{"id":"2402.03220","title":"The Benefits of Reusing Batches for Gradient Descent in Two-Layer Networks: Breaking the Curse of Information and Leap Exponents","created":"2024-02-05","cited_by_count":52,"area":null},{"id":"2212.07469","title":"Learning threshold neurons via the \"edge of stability\"","created":"2022-12-14","cited_by_count":51,"area":null},{"id":"2202.00293","title":"Phase diagram of Stochastic Gradient Descent in high-dimensional two-layer neural networks","created":"2022-02-01","cited_by_count":50,"area":null},{"id":"2409.14623","title":"From Lazy to Rich: Exact Learning Dynamics in Deep Linear Networks","created":"2024-09-22","cited_by_count":49,"area":null},{"id":"2102.02410","title":"A Local Convergence Theory for Mildly Over-Parameterized Two-Layer Neural Network","created":"2021-02-04","cited_by_count":47,"area":null},{"id":"2306.17759","title":"The Shaped Transformer: Attention Models in the Infinite Depth-and-Width Limit","created":"2023-06-30","cited_by_count":47,"area":null},{"id":"2211.08775","title":"Unbalanced Optimal Transport, from Theory to Numerics","created":"2022-11-16","cited_by_count":46,"area":null},{"id":"2402.01258","title":"Transformers Learn Nonlinear Features In Context: Nonconvex Mean-field Dynamics on the Attention Landscape","created":"2024-02-02","cited_by_count":46,"area":null},{"id":"2101.09612","title":"On the Proof of Global Convergence of Gradient Descent for Deep ReLU Networks with Linear Widths","created":"2021-01-24","cited_by_count":44,"area":null},{"id":"2106.03795","title":"Heavy Tails in SGD and Compressibility of Overparametrized Neural Networks","created":"2021-06-07","cited_by_count":44,"area":null},{"id":"2302.05882","title":"From high-dimensional & mean-field dynamics to dimensionless ODEs: A unifying approach to SGD in two-layers networks","created":"2023-02-12","cited_by_count":43,"area":null},{"id":"2304.03408","title":"Dynamics of Finite Width Kernel and Prediction Fluctuations in Mean Field Neural Networks","created":"2023-04-06","cited_by_count":43,"area":null},{"id":"2406.06158","title":"Get rich quick: exact solutions reveal how unbalanced initializations promote rapid feature learning","created":"2024-06-10","cited_by_count":42,"area":null},{"id":"2206.02768","title":"The Neural Covariance SDE: Shaped Infinite Depth-and-Width Networks at Initialization","created":"2022-06-06","cited_by_count":41,"area":null},{"id":"2305.18411","title":"Feature-Learning Networks Are Consistent Across Widths At Realistic Scales","created":"2023-05-28","cited_by_count":41,"area":null},{"id":"2106.04546","title":"LEADS: Learning Dynamical Systems that Generalize Across Environments","created":"2021-06-08","cited_by_count":40,"area":null},{"id":"2307.11353","title":"What can a Single Attention Layer Learn? A Study Through the Random Features Lens","created":"2023-07-21","cited_by_count":40,"area":null},{"id":"2405.15712","title":"Infinite Limits of Multi-head Transformer Dynamics","created":"2024-05-24","cited_by_count":39,"area":null},{"id":"2410.23228","title":"Emergence of meta-stable clustering in mean-field transformer models","created":"2024-10-30","cited_by_count":39,"area":null},{"id":"2105.14573","title":"Embedding Principle of Loss Landscape of Deep Neural Networks","created":"2021-05-30","cited_by_count":38,"area":null},{"id":"2209.15055","title":"Implicit Bias of Large Depth Networks: a Notion of Rank for Nonlinear Functions","created":"2022-09-29","cited_by_count":38,"area":null},{"id":"2311.14975","title":"Eliminating Domain Bias for Federated Learning in Representation Space","created":"2023-11-25","cited_by_count":38,"area":null},{"id":"2402.04980","title":"Asymptotics of feature learning in two-layer networks after one gradient-step","created":"2024-02-07","cited_by_count":38,"area":null},{"id":"2106.08929","title":"KALE Flow: A Relaxed KL Gradient Flow for Probabilities with Disjoint Support","created":"2021-06-16","cited_by_count":37,"area":null},{"id":"2301.12832","title":"Deep networks for system identification: a Survey","created":"2023-01-30","cited_by_count":37,"area":null},{"id":"2405.15459","title":"Repetita Iuvant: Data Repetition Allows SGD to Learn High-Dimensional Multi-Index Functions","created":"2024-05-24","cited_by_count":37,"area":null},{"id":"2202.07626","title":"Random Feature Amplification: Feature Learning and Generalization in Neural Networks","created":"2022-02-15","cited_by_count":35,"area":null},{"id":"2206.08873","title":"Mirror Descent with Relative Smoothness in Measure Spaces, with application to Sinkhorn and EM","created":"2022-06-17","cited_by_count":34,"area":null},{"id":"2304.06879","title":"Performative Prediction with Neural Networks","created":"2023-04-14","cited_by_count":34,"area":null},{"id":"2106.04013","title":"The Future is Log-Gaussian: ResNets and Their Infinite-Depth-and-Width Limit at Initialization","created":"2021-06-07","cited_by_count":33,"area":null},{"id":"2105.11686","title":"Towards Understanding the Condensation of Neural Networks at Initial Training","created":"2021-05-25","cited_by_count":32,"area":null},{"id":"2309.03843","title":"Gradient-Based Feature Learning under Structured Data","created":"2023-09-07","cited_by_count":32,"area":null},{"id":"2211.06506","title":"Spectral Evolution and Invariance in Linear-width Neural Networks","created":"2022-11-11","cited_by_count":30,"area":null},{"id":"2206.09841","title":"Label noise (stochastic) gradient descent implicitly solves the Lasso for quadratic parametrisation","created":"2022-06-20","cited_by_count":29,"area":null},{"id":"2211.05641","title":"Regression as Classification: Influence of Task Formulation on Neural Network Features","created":"2022-11-10","cited_by_count":29,"area":null},{"id":"2604.21691","title":"There Will Be a Scientific Theory of Deep Learning","created":"2026-04-23","cited_by_count":29,"area":null},{"id":"2301.01766","title":"Learning Gaussian Mixtures Using the Wasserstein-Fisher-Rao Gradient Flow","created":"2023-01-04","cited_by_count":27,"area":null},{"id":"2305.05080","title":"Scalable Optimal Transport Methods in Machine Learning: A Contemporary Survey","created":"2023-05-08","cited_by_count":27,"area":null},{"id":"2105.02095","title":"Two-layer neural networks with values in a Banach space","created":"2021-05-05","cited_by_count":26,"area":null},{"id":"2206.04613","title":"Explicit Regularization in Overparametrized Models via Noise Injection","created":"2022-06-09","cited_by_count":26,"area":null},{"id":"2302.00457","title":"Simplicity Bias in 1-Hidden Layer Neural Networks","created":"2023-02-01","cited_by_count":26,"area":null},{"id":"2106.02588","title":"Stochastic gradient descent with noise of machine learning type. Part II: Continuous time analysis","created":"2021-06-04","cited_by_count":25,"area":null},{"id":"2109.03839","title":"Sqrt(d) Dimension Dependence of Langevin Monte Carlo","created":"2021-09-08","cited_by_count":25,"area":null},{"id":"2302.10034","title":"Over-Parameterization Exponentially Slows Down Gradient Descent for Learning a Single Neuron","created":"2023-02-20","cited_by_count":25,"area":null},{"id":"2310.19103","title":"Proving Linear Mode Connectivity of Neural Networks via Optimal Transport","created":"2023-10-29","cited_by_count":25,"area":null},{"id":"2509.25040","title":"A multiscale analysis of mean-field transformers in the moderate interaction regime","created":"2025-09-29","cited_by_count":25,"area":null},{"id":"2105.14417","title":"Overparameterization of deep ResNet: zero loss and mean-field analysis","created":"2021-05-30","cited_by_count":24,"area":null},{"id":"2107.13289","title":"The loss landscape of deep linear neural networks: a second-order analysis","created":"2021-07-28","cited_by_count":24,"area":null},{"id":"2203.16462","title":"Convergence of gradient descent for deep neural networks","created":"2022-03-30","cited_by_count":24,"area":null},{"id":"2310.09753","title":"When can transformers reason with abstract symbols?","created":"2023-10-15","cited_by_count":24,"area":null},{"id":"2212.03050","title":"Uniform-in-time propagation of chaos for mean field Langevin dynamics","created":"2022-12-06","cited_by_count":23,"area":null},{"id":"2402.17457","title":"Super Consistency of Neural Network Landscapes and Learning Rate Transfer","created":"2024-02-27","cited_by_count":23,"area":null},{"id":"2111.15527","title":"Embedding Principle: a hierarchical structure of loss landscape of deep neural networks","created":"2021-11-30","cited_by_count":22,"area":null},{"id":"2106.06529","title":"The Limitations of Large Width in Neural Networks: A Deep Gaussian Process Perspective","created":"2021-06-11","cited_by_count":21,"area":null},{"id":"2106.13792","title":"Proxy Convexity: A Unified Framework for the Analysis of Neural Networks Trained by Gradient Descent","created":"2021-06-25","cited_by_count":21,"area":null},{"id":"2201.07296","title":"Convergence of Policy Gradient for Entropy Regularized MDPs with Neural Network Approximation in the Mean-Field Regime","created":"2022-01-18","cited_by_count":21,"area":null},{"id":"2204.06274","title":"Overparameterized Linear Regression under Adversarial Attacks","created":"2022-04-13","cited_by_count":21,"area":null},{"id":"2212.14457","title":"Bayesian Interpolation with Deep Linear Networks","created":"2022-12-29","cited_by_count":21,"area":null},{"id":"2106.06573","title":"Understanding Deflation Process in Over-parametrized Tensor Decomposition","created":"2021-06-11","cited_by_count":20,"area":null},{"id":"2211.16980","title":"Infinite-width limit of deep linear neural networks","created":"2022-11-29","cited_by_count":20,"area":null},{"id":"2306.07221","title":"Convergence of mean-field Langevin dynamics: Time and space discretization, stochastic gradient, and variance reduction","created":"2023-06-12","cited_by_count":20,"area":null},{"id":"2307.06092","title":"Quantitative CLTs in Deep Neural Networks","created":"2023-07-12","cited_by_count":20,"area":null},{"id":"2308.13431","title":"Six Lectures on Linearized Neural Networks","created":"2023-08-25","cited_by_count":20,"area":null},{"id":"2310.03597","title":"Sampling via Gradient Flows in the Space of Probability Measures","created":"2023-10-05","cited_by_count":20,"area":null},{"id":"2401.07187","title":"A Survey on Statistical Theory of Deep Learning: Approximation, Training Dynamics, and Generative Models","created":"2024-01-14","cited_by_count":20,"area":null},{"id":"2501.08537","title":"Complexity Control Facilitates Reasoning-Based Compositional Generalization in Transformers","created":"2025-01-15","cited_by_count":20,"area":null},{"id":"2101.02533","title":"Towards Understanding Learning in Neural Networks with Linear Teachers","created":"2021-01-07","cited_by_count":19,"area":null},{"id":"2104.05508","title":"Noether: The More Things Change, the More Stay the Same","created":"2021-04-12","cited_by_count":19,"area":null},{"id":"2107.09133","title":"The Limiting Dynamics of SGD: Modified Loss, Phase Space Oscillations, and Anomalous Diffusion","created":"2021-07-19","cited_by_count":19,"area":null},{"id":"2210.04909","title":"Meta-Principled Family of Hyperparameter Scaling Strategies","created":"2022-10-10","cited_by_count":19,"area":null},{"id":"2304.09576","title":"Leveraging the two timescale regime to demonstrate convergence of neural networks","created":"2023-04-19","cited_by_count":19,"area":null},{"id":"2307.11807","title":"Local Kernel Renormalization as a mechanism for feature learning in overparametrized Convolutional Neural Networks","created":"2023-07-21","cited_by_count":19,"area":null},{"id":"2101.01708","title":"A Priori Generalization Analysis of the Deep Ritz Method for Solving High Dimensional Elliptic Equations","created":"2021-01-05","cited_by_count":18,"area":null},{"id":"2206.00860","title":"Self-Consistency of the Fokker-Planck Equation","created":"2022-06-02","cited_by_count":18,"area":null},{"id":"2209.07263","title":"Robustness in deep learning: The good (width), the bad (depth), and the ugly (initialization)","created":"2022-09-15","cited_by_count":18,"area":null}],"next_cursor":"100"}