{"total":142,"rows":[{"id":"2206.15144","title":"Neural Networks can Learn Representations with Gradient Descent","created":"2022-06-30","cited_by_count":161,"area":null},{"id":"2205.01445","title":"High-dimensional Asymptotics of Feature Learning: How One Gradient Step Improves the Representation","created":"2022-05-03","cited_by_count":150,"area":null},{"id":"2302.11055","title":"SGD learning on neural networks: leap complexity and saddle-to-saddle dynamics","created":"2023-02-21","cited_by_count":143,"area":null},{"id":"2411.04330","title":"Scaling Laws for Precision","created":"2024-11-07","cited_by_count":95,"area":null},{"id":"2210.15651","title":"Learning Single-Index Models with Shallow Neural Networks","created":"2022-10-27","cited_by_count":92,"area":null},{"id":"2301.13105","title":"Generalization on the Unseen, Logic Reasoning and Degree Curriculum","created":"2023-01-30","cited_by_count":73,"area":null},{"id":"2305.18270","title":"How Two-Layer Neural Networks Learn, One (Giant) Step at a Time","created":"2023-05-29","cited_by_count":68,"area":null},{"id":"2209.14863","title":"Neural Networks Efficiently Learn Low-Dimensional Representations with SGD","created":"2022-09-29","cited_by_count":63,"area":null},{"id":"2305.10633","title":"Smoothing the Landscape Boosts the Signal for SGD: Optimal Sample Complexity for Learning Single Index Models","created":"2023-05-18","cited_by_count":61,"area":null},{"id":"2303.00055","title":"Learning time-scales in two-layers neural networks","created":"2023-02-28","cited_by_count":57,"area":null},{"id":"2310.19793","title":"On Learning Gaussian Multi-index Models with Gradient Flow","created":"2023-10-30","cited_by_count":57,"area":null},{"id":"2306.07042","title":"Transformers learn through gradual rank increase","created":"2023-06-12","cited_by_count":52,"area":null},{"id":"2402.03220","title":"The Benefits of Reusing Batches for Gradient Descent in Two-Layer Networks: Breaking the Curse of Information and Leap Exponents","created":"2024-02-05","cited_by_count":52,"area":null},{"id":"2306.17759","title":"The Shaped Transformer: Attention Models in the Infinite Depth-and-Width Limit","created":"2023-06-30","cited_by_count":47,"area":null},{"id":"2406.01581","title":"Neural network learns low-dimensional polynomials with SGD near the information-theoretic limit","created":"2024-06-03","cited_by_count":46,"area":null},{"id":"2403.05529","title":"Computational-Statistical Gaps in Gaussian Single-Index Models","created":"2024-03-08","cited_by_count":44,"area":null},{"id":"2302.05882","title":"From high-dimensional & mean-field dynamics to dimensionless ODEs: A unifying approach to SGD in two-layers networks","created":"2023-02-12","cited_by_count":43,"area":null},{"id":"2204.02892","title":"Sub-Task Decomposition Enables Learning in Sequence to Sequence Tasks","created":"2022-04-06","cited_by_count":41,"area":null},{"id":"2307.11353","title":"What can a Single Attention Layer Learn? A Study Through the Random Features Lens","created":"2023-07-21","cited_by_count":40,"area":null},{"id":"2309.15111","title":"SGD Finds then Tunes Features in Two-Layer Neural Networks with near-Optimal Sample Complexity: A Case Study in the XOR problem","created":"2023-09-26","cited_by_count":39,"area":null},{"id":"2402.04980","title":"Asymptotics of feature learning in two-layer networks after one gradient-step","created":"2024-02-07","cited_by_count":38,"area":null},{"id":"2504.10957","title":"When is Task Vector Provably Effective for Model Editing? A Generalization Analysis of Nonlinear Transformers","created":"2025-04-15","cited_by_count":38,"area":"safety"},{"id":"2405.15459","title":"Repetita Iuvant: Data Repetition Allows SGD to Learn High-Dimensional Multi-Index Functions","created":"2024-05-24","cited_by_count":37,"area":null},{"id":"2406.06467","title":"How Far Can Transformers Reason? The Globality Barrier and Inductive Scratchpad","created":"2024-06-10","cited_by_count":36,"area":null},{"id":"2309.03843","title":"Gradient-Based Feature Learning under Structured Data","created":"2023-09-07","cited_by_count":32,"area":null},{"id":"2406.02128","title":"Iteration Head: A Mechanistic Study of Chain-of-Thought","created":"2024-06-04","cited_by_count":31,"area":"lr"},{"id":"2411.02544","title":"Pretrained transformer efficiently learns low-dimensional target functions in-context","created":"2024-11-04","cited_by_count":31,"area":null},{"id":"2410.05459","title":"From Sparse Dependence to Sparse Attention: Unveiling How Chain-of-Thought Enhances Transformer Sample Efficiency","created":"2024-10-07","cited_by_count":29,"area":"lr"},{"id":"2604.21691","title":"There Will Be a Scientific Theory of Deep Learning","created":"2026-04-23","cited_by_count":29,"area":null},{"id":"2402.10127","title":"Nonlinear spiked covariance matrices and signal propagation in deep neural networks","created":"2024-02-15","cited_by_count":26,"area":null},{"id":"2212.13881","title":"Mechanism of feature learning in deep fully connected networks and kernel machines that recursively learn features","created":"2022-12-28","cited_by_count":25,"area":null},{"id":"2505.17863","title":"The emergence of sparse attention: impact of data distribution and benefits of repetition","created":"2025-05-23","cited_by_count":25,"area":null},{"id":"2306.16921","title":"Provable Advantage of Curriculum Learning on Parity Targets with Mixed Inputs","created":"2023-06-29","cited_by_count":22,"area":null},{"id":"2306.07221","title":"Convergence of mean-field Langevin dynamics: Time and space discretization, stochastic gradient, and variance reduction","created":"2023-06-12","cited_by_count":20,"area":null},{"id":"2308.13431","title":"Six Lectures on Linearized Neural Networks","created":"2023-08-25","cited_by_count":20,"area":null},{"id":"2301.13833","title":"A Mathematical Model for Curriculum Learning for Parities","created":"2023-01-31","cited_by_count":19,"area":null},{"id":"2305.06986","title":"Provable Guarantees for Nonlinear Feature Learning in Three-Layer Neural Networks","created":"2023-05-11","cited_by_count":19,"area":null},{"id":"2405.15480","title":"Fundamental computational limits of weak learnability in high-dimensional multi-index models","created":"2024-05-24","cited_by_count":19,"area":null},{"id":"2208.03113","title":"On the non-universality of deep learning: quantifying the cost of symmetry","created":"2022-08-05","cited_by_count":18,"area":null},{"id":"2405.17580","title":"Mixed Dynamics In Linear Networks: Unifying the Lazy and Active Regimes","created":"2024-05-27","cited_by_count":18,"area":null},{"id":"2205.13647","title":"Learning to Reason with Neural Networks: Generalization, Unseen Data and Boolean Measures","created":"2022-05-26","cited_by_count":17,"area":null},{"id":"2307.15804","title":"On Single Index Models beyond Gaussian Data","created":"2023-07-28","cited_by_count":17,"area":null},{"id":"2208.01003","title":"What Can Be Learnt With Wide Convolutional Neural Networks?","created":"2022-08-01","cited_by_count":16,"area":null},{"id":"2307.06887","title":"Provable Multi-Task Representation Learning by Two-Layer ReLU Neural Networks","created":"2023-07-13","cited_by_count":16,"area":null},{"id":"2401.04553","title":"Linear Recursive Feature Machines provably recover low-rank matrices","created":"2024-01-09","cited_by_count":16,"area":null},{"id":"2410.04887","title":"Wide Neural Networks Trained with Weight Decay Provably Exhibit Neural Collapse","created":"2024-10-07","cited_by_count":16,"area":null},{"id":"2410.18938","title":"A Random Matrix Theory Perspective on the Spectrum of Learned Features and Asymptotic Generalization Capabilities","created":"2024-10-24","cited_by_count":16,"area":null},{"id":"2505.17495","title":"ProxySPEX: Inference-Efficient Interpretability via Sparse Feature Interactions in LLMs","created":"2025-05-23","cited_by_count":16,"area":null},{"id":"2508.10053","title":"xRFM: Accurate, scalable, and interpretable feature learning models for tabular data","created":"2025-08-12","cited_by_count":16,"area":null},{"id":"2305.18502","title":"Escaping mediocrity: how two-layer networks learn hard generalized linear models with SGD","created":"2023-05-29","cited_by_count":15,"area":null},{"id":"2403.09053","title":"Towards a theory of model distillation","created":"2024-03-14","cited_by_count":15,"area":null},{"id":"2407.05622","title":"On the Complexity of Learning Sparse Functions with Statistical and Gradient Queries","created":"2024-07-08","cited_by_count":15,"area":null},{"id":"2408.07254","title":"Learning Multi-Index Models with Neural Networks via Mean-Field Langevin Dynamics","created":"2024-08-14","cited_by_count":15,"area":null},{"id":"2410.05464","title":"Progressive distillation induces an implicit curriculum","created":"2024-10-07","cited_by_count":15,"area":null},{"id":"2504.05426","title":"Survey on Algorithms for multi-index models","created":"2025-04-07","cited_by_count":15,"area":null},{"id":"2208.02789","title":"Feature selection with gradient descent on two-layer networks in low-rotation regimes","created":"2022-08-04","cited_by_count":14,"area":null},{"id":"2310.12408","title":"Provable Guarantees for Neural Networks via Gradient Feature Learning","created":"2023-10-19","cited_by_count":14,"area":null},{"id":"2506.06489","title":"Alternating Gradient Flows: A Theory of Feature Learning in Two-layer Neural Networks","created":"2025-06-06","cited_by_count":14,"area":null},{"id":"2507.02550","title":"Position: A Theory of Deep Learning Must Include Compositional Sparsity","created":"2025-07-03","cited_by_count":14,"area":null},{"id":"2309.03800","title":"Pareto Frontiers in Neural Feature Learning: Data, Compute, Width, and Luck","created":"2023-09-07","cited_by_count":13,"area":null},{"id":"2403.14917","title":"Mean-field Analysis on Two-layer Neural Networks from a Kernel Perspective","created":"2024-03-22","cited_by_count":13,"area":null},{"id":"2403.03375","title":"Complexity Matters: Dynamics of Feature Learning in the Presence of Spurious Correlations","created":"2024-03-05","cited_by_count":12,"area":null},{"id":"2404.08602","title":"Sliding down the stairs: how correlated latent variables accelerate learning with neural networks","created":"2024-04-12","cited_by_count":12,"area":null},{"id":"2410.11275","title":"Shallow diffusion networks provably learn hidden low-dimensional structure","created":"2024-10-15","cited_by_count":12,"area":"diffusion"},{"id":"2306.16308","title":"Gaussian random field approximation via Stein's method with applications to wide random neural networks","created":"2023-06-28","cited_by_count":11,"area":null},{"id":"2405.09312","title":"Agnostic Active Learning of Single Index Models with Linear Sample Complexity","created":"2024-05-15","cited_by_count":11,"area":null},{"id":"2205.14818","title":"Excess Risk of Two-Layer ReLU Neural Networks in Teacher-Student Settings and its Superiority to Kernel Methods","created":"2022-05-30","cited_by_count":10,"area":"distill"},{"id":"2402.05271","title":"Feature learning as alignment: a structural property of gradient descent in non-linear neural networks","created":"2024-02-07","cited_by_count":10,"area":null},{"id":"2402.05913","title":"Efficient Stagewise Pretraining via Progressive Subnetworks","created":"2024-02-08","cited_by_count":10,"area":null},{"id":"2311.13774","title":"Learning Hierarchical Polynomials with Three-Layer Neural Networks","created":"2023-11-23","cited_by_count":9,"area":null},{"id":"2506.02651","title":"Asymptotics of SGD in Sequence-Single Index Models and Single-Layer Attention Networks","created":"2025-06-03","cited_by_count":9,"area":null},{"id":"2506.05500","title":"The Generative Leap: Sharp Sample Complexity for Efficiently Learning Gaussian Multi-Index Models","created":"2025-06-05","cited_by_count":9,"area":null},{"id":"2211.08771","title":"On the symmetries in the dynamics of wide two-layer neural networks","created":"2022-11-16","cited_by_count":8,"area":null},{"id":"2404.18769","title":"Learning with Norm Constrained, Over-parameterized, Two-layer Neural Networks","created":"2024-04-29","cited_by_count":8,"area":null},{"id":"2406.11110","title":"How Neural Networks Learn the Support is an Implicit Regularization Effect of SGD","created":"2024-06-17","cited_by_count":8,"area":null},{"id":"2406.17167","title":"Learning on Transformers is Provable Low-Rank and Sparse: A One-layer Analysis","created":"2024-06-24","cited_by_count":8,"area":null},{"id":"2502.06443","title":"Low-dimensional Functions are Efficiently Learnable under Randomly Biased Distributions","created":"2025-02-10","cited_by_count":8,"area":null},{"id":"2410.10024","title":"Sharper Guarantees for Learning Neural Network Classifiers with Gradient Methods","created":"2024-10-13","cited_by_count":7,"area":null},{"id":"2502.02545","title":"Optimal Spectral Transitions in High-Dimensional Multi-Index Models","created":"2025-02-04","cited_by_count":7,"area":null},{"id":"2512.22768","title":"Understanding the Mechanisms of Fast Hyperparameter Transfer","created":"2025-12-28","cited_by_count":7,"area":null},{"id":"2304.00707","title":"High-dimensional scaling limits and fluctuations of online least-squares SGD with smooth covariance","created":"2023-04-03","cited_by_count":6,"area":null},{"id":"2407.05664","title":"How DNNs break the Curse of Dimensionality: Compositionality and Symmetry Learning","created":"2024-07-08","cited_by_count":6,"area":null},{"id":"2506.02664","title":"Computational Thresholds in Multi-Modal Learning via the Spiked Matrix-Tensor Model","created":"2025-06-03","cited_by_count":6,"area":null},{"id":"2509.24914","title":"Single-Head Attention in High Dimensions: A Theory of Generalization, Weights Spectra, and Scaling Laws","created":"2025-09-29","cited_by_count":6,"area":null},{"id":"2601.22652","title":"Spectral Gradient Descent Mitigates Anisotropy-Driven Misalignment: A Case Study in Phase Retrieval","created":"2026-01-30","cited_by_count":6,"area":null},{"id":"2310.02117","title":"Symmetric Single Index Learning","created":"2023-10-03","cited_by_count":5,"area":null},{"id":"2310.03696","title":"Function-Space Optimality of Neural Architectures with Multivariate Nonlinearities","created":"2023-10-05","cited_by_count":5,"area":null},{"id":"2310.05309","title":"Optimizing Solution-Samplers for Combinatorial Problems: The Landscape of Policy-Gradient Methods","created":"2023-10-08","cited_by_count":5,"area":null},{"id":"2402.05013","title":"Compression of Structured Data with Autoencoders: Provable Benefit of Nonlinearities and Depth","created":"2024-02-07","cited_by_count":5,"area":null},{"id":"2410.16449","title":"Robust Feature Learning for Multi-Index Models in High Dimensions","created":"2024-10-21","cited_by_count":5,"area":null},{"id":"2602.02285","title":"AI4SLT: Empirical Processes in Lean 4 for Formal Statistical Learning Theory","created":"2026-02-02","cited_by_count":5,"area":null},{"id":"2602.10867","title":"Deep Learning of Compositional Targets with Hierarchical Spectral Methods","created":"2026-02-11","cited_by_count":5,"area":null},{"id":"2605.06258","title":"The Weight Gram Matrix Captures Sequential Feature Linearization in Deep Networks","created":"2026-05-07","cited_by_count":5,"area":null},{"id":"2303.02957","title":"Primal and Dual Analysis of Entropic Fictitious Play for Finite-sum Problems","created":"2023-03-06","cited_by_count":4,"area":null},{"id":"2310.12079","title":"Differential Equation Scaling Limits of Shaped and Unshaped Neural Networks","created":"2023-10-18","cited_by_count":4,"area":null},{"id":"2401.07085","title":"Three Mechanisms of Feature Learning in a Linear Network","created":"2024-01-13","cited_by_count":4,"area":null},{"id":"2406.01766","title":"How Does Gradient Descent Learn Features -- A Local Analysis for Regularized Two-Layer Neural Networks","created":"2024-06-03","cited_by_count":4,"area":null},{"id":"2407.20724","title":"Exploring Loss Landscapes through the Lens of Spin Glass Theory","created":"2024-07-30","cited_by_count":4,"area":null},{"id":"2505.21722","title":"Saddle-To-Saddle Dynamics in Deep ReLU Networks: Low-Rank Bias in the First Saddle Escape","created":"2025-05-27","cited_by_count":4,"area":null},{"id":"2406.02157","title":"Online Learning and Information Exponents: On The Importance of Batch size, and Time/Complexity Tradeoffs","created":"2024-06-04","cited_by_count":3,"area":null}],"next_cursor":"100"}