{"total":118,"rows":[{"id":"2207.08799","title":"Hidden Progress in Deep Learning: SGD Learns Parities Near the Computational Limit","created":"2022-07-18","cited_by_count":187,"area":null},{"id":"2205.01445","title":"High-dimensional Asymptotics of Feature Learning: How One Gradient Step Improves the Representation","created":"2022-05-03","cited_by_count":150,"area":null},{"id":"2203.12221","title":"Modality Competition: What Makes Joint Training of Multi-modal Network Fail in Deep Learning? (Provably)","created":"2022-03-23","cited_by_count":122,"area":null},{"id":"2202.06526","title":"Benign Overfitting in Two-layer Convolutional Neural Networks","created":"2022-02-14","cited_by_count":91,"area":null},{"id":"2110.13905","title":"Gradient Descent on Two-layer Nets: Margin Maximization and Simplicity Bias","created":"2021-10-26","cited_by_count":83,"area":null},{"id":"2311.18817","title":"Dichotomy of Early and Late Phase Implicit Biases Can Provably Induce Grokking","created":"2023-11-30","cited_by_count":80,"area":null},{"id":"2202.05928","title":"Benign Overfitting without Linearity: Neural Network Classifiers Trained by Gradient Descent for Noisy Linear Data","created":"2022-02-11","cited_by_count":74,"area":null},{"id":"2207.12678","title":"Analyzing Sharpness along GD Trajectory: Progressive Sharpening and Edge of Stability","created":"2022-07-26","cited_by_count":63,"area":null},{"id":"2108.11371","title":"Understanding the Generalization of Adam in Learning Neural Networks with Proper Regularization","created":"2021-08-25","cited_by_count":49,"area":null},{"id":"2201.01212","title":"AutoBalance: Optimized Loss Functions for Imbalanced Data","created":"2022-01-04","cited_by_count":49,"area":null},{"id":"2101.09612","title":"On the Proof of Global Convergence of Gradient Descent for Deep ReLU Networks with Linear Widths","created":"2021-01-24","cited_by_count":44,"area":null},{"id":"2204.02892","title":"Sub-Task Decomposition Enables Learning in Sequence to Sequence Tasks","created":"2022-04-06","cited_by_count":41,"area":null},{"id":"2206.02768","title":"The Neural Covariance SDE: Shaped Infinite Depth-and-Width Networks at Initialization","created":"2022-06-06","cited_by_count":41,"area":null},{"id":"2310.12680","title":"On the Optimization and Generalization of Multi-head Attention","created":"2023-10-19","cited_by_count":37,"area":null},{"id":"2205.10217","title":"Memorization and Optimization in Deep Neural Networks with Minimum Over-parameterization","created":"2022-05-20","cited_by_count":36,"area":null},{"id":"2106.05932","title":"Early-stopped neural networks are consistent","created":"2021-06-10","cited_by_count":34,"area":null},{"id":"2306.03534","title":"Continual Learning in Linear Classification on Separable Data","created":"2023-06-06","cited_by_count":34,"area":null},{"id":"2110.04622","title":"Does Preprocessing Help Training Over-parameterized Neural Networks?","created":"2021-10-09","cited_by_count":32,"area":null},{"id":"2211.06506","title":"Spectral Evolution and Invariance in Linear-width Neural Networks","created":"2022-11-11","cited_by_count":30,"area":null},{"id":"2301.13139","title":"A Novel Framework for Policy Mirror Descent with General Parameterization and Linear Convergence","created":"2023-01-30","cited_by_count":30,"area":null},{"id":"2309.07418","title":"A Fast Optimization View: Reformulating Single Layer Attention in LLM Based on Tensor and SVM Trick, and Solving It in Matrix Multiplication Time","created":"2023-09-14","cited_by_count":30,"area":null},{"id":"2211.14227","title":"Bypass Exponential Time Preprocessing: Fast Neural Network Training via Weight-Data Correlation Preprocessing","created":"2022-11-25","cited_by_count":29,"area":null},{"id":"2105.10585","title":"Properties of the After Kernel","created":"2021-05-21","cited_by_count":27,"area":null},{"id":"2111.01875","title":"Subquadratic Overparameterization for Shallow Neural Networks","created":"2021-11-02","cited_by_count":26,"area":null},{"id":"2308.12247","title":"How to Protect Copyright Data in Optimization of Large Language Models?","created":"2023-08-23","cited_by_count":26,"area":null},{"id":"2106.13792","title":"Proxy Convexity: A Unified Framework for the Analysis of Neural Networks Trained by Gradient Descent","created":"2021-06-25","cited_by_count":21,"area":null},{"id":"2302.09235","title":"Generalization and Stability of Interpolating Neural Networks with Minimal Width","created":"2023-02-18","cited_by_count":21,"area":null},{"id":"2206.00833","title":"Finite-Time Analysis of Entropy-Regularized Neural Natural Actor-Critic Algorithm","created":"2022-06-02","cited_by_count":20,"area":null},{"id":"2308.13431","title":"Six Lectures on Linearized Neural Networks","created":"2023-08-25","cited_by_count":20,"area":null},{"id":"2101.02533","title":"Towards Understanding Learning in Neural Networks with Linear Teachers","created":"2021-01-07","cited_by_count":19,"area":null},{"id":"2101.01152","title":"Provable Generalization of SGD-trained Neural Networks of Any Width in the Presence of Adversarial Label Noise","created":"2021-01-04","cited_by_count":17,"area":null},{"id":"2102.04394","title":"Learning with Density Matrices and Random Features","created":"2021-02-08","cited_by_count":17,"area":null},{"id":"2205.11787","title":"Quadratic models for understanding catapult dynamics of neural networks","created":"2022-05-24","cited_by_count":17,"area":null},{"id":"2101.04968","title":"Learning with Gradient Descent and Weakly Convex Losses","created":"2021-01-13","cited_by_count":16,"area":null},{"id":"2307.06887","title":"Provable Multi-Task Representation Learning by Two-Layer ReLU Neural Networks","created":"2023-07-13","cited_by_count":16,"area":null},{"id":"2206.12802","title":"Bounding the Width of Neural Networks via Coupled Initialization -- A Worst Case Analysis","created":"2022-06-26","cited_by_count":15,"area":null},{"id":"2305.12467","title":"Understanding Multi-phase Optimization Dynamics and Rich Nonlinear Behaviors of ReLU Networks","created":"2023-05-21","cited_by_count":15,"area":null},{"id":"2108.00207","title":"The Separation Capacity of Random Neural Networks","created":"2021-07-31","cited_by_count":14,"area":null},{"id":"2112.03657","title":"Understanding Square Loss in Training Overparametrized Neural Network Classifiers","created":"2021-12-07","cited_by_count":14,"area":null},{"id":"2208.02789","title":"Feature selection with gradient descent on two-layer networks in low-rotation regimes","created":"2022-08-04","cited_by_count":14,"area":null},{"id":"2306.13853","title":"A Unified Approach to Controlling Implicit Regularization via Mirror Descent","created":"2023-06-24","cited_by_count":14,"area":null},{"id":"2407.13621","title":"Differential Privacy Mechanisms in Neural Tangent Kernel Regression","created":"2024-07-18","cited_by_count":14,"area":null},{"id":"2103.05243","title":"On the Generalization Power of Overfitted Two-Layer Neural Tangent Kernel Models","created":"2021-03-09","cited_by_count":13,"area":null},{"id":"2110.11280","title":"Actor-critic is implicitly biased towards high entropy optimal policies","created":"2021-10-21","cited_by_count":13,"area":null},{"id":"2309.13482","title":"A Unified Scheme of ResNet and Softmax","created":"2023-09-23","cited_by_count":12,"area":null},{"id":"2310.12462","title":"Unmasking Transformers: A Theoretical Approach to Data Recovery via Attention Weights","created":"2023-10-19","cited_by_count":12,"area":null},{"id":"2105.06569","title":"The Dynamics of Gradient Descent for Overparametrized Neural Networks","created":"2021-05-13","cited_by_count":11,"area":null},{"id":"2112.02668","title":"On the Convergence of Shallow Neural Network Training with Randomly Masked Neurons","created":"2021-12-05","cited_by_count":11,"area":null},{"id":"2206.02927","title":"Spectral Bias Outside the Training Set for Deep Networks in the Kernel Regime","created":"2022-06-06","cited_by_count":11,"area":null},{"id":"2402.03625","title":"Convex Relaxations of ReLU Neural Networks Approximate Global Optima in Polynomial Time","created":"2024-02-06","cited_by_count":11,"area":null},{"id":"2406.00588","title":"Generalization Bound and New Algorithm for Clean-Label Backdoor Attack","created":"2024-06-02","cited_by_count":11,"area":null},{"id":"2103.01391","title":"Sample Complexity and Overparameterization Bounds for Temporal Difference Learning with Neural Network Approximation","created":"2021-03-02","cited_by_count":10,"area":null},{"id":"2203.14328","title":"On the Neural Tangent Kernel Analysis of Randomly Pruned Neural Networks","created":"2022-03-27","cited_by_count":9,"area":null},{"id":"2310.11291","title":"An Automatic Learning Rate Schedule Algorithm for Achieving Faster Convergence and Steeper Descent","created":"2023-10-17","cited_by_count":9,"area":null},{"id":"2311.15390","title":"Local Convergence of Approximate Newton Method for Two Layer Nonlinear Regression","created":"2023-11-26","cited_by_count":9,"area":null},{"id":"2502.13283","title":"Benefits of Early Stopping in Gradient Descent for Overparameterized Logistic Regression","created":"2025-02-18","cited_by_count":9,"area":null},{"id":"2110.05960","title":"Imitating Deep Learning Dynamics via Locally Elastic Stochastic Differential Equations","created":"2021-10-11","cited_by_count":8,"area":null},{"id":"2210.12001","title":"When Expressivity Meets Trainability: Fewer than $n$ Neurons Can Work","created":"2022-10-21","cited_by_count":8,"area":null},{"id":"2402.12241","title":"Convergence of Gradient Descent for Recurrent Neural Networks: A Nonasymptotic Analysis","created":"2024-02-19","cited_by_count":8,"area":null},{"id":"2209.08399","title":"Approximation results for Gradient Descent trained Shallow Neural Networks in $1d$","created":"2022-09-17","cited_by_count":7,"area":null},{"id":"2209.15106","title":"Restricted Strong Convexity of Deep Learning Models with Smooth Activations","created":"2022-09-29","cited_by_count":7,"area":null},{"id":"2410.10024","title":"Sharper Guarantees for Learning Neural Network Classifiers with Gradient Methods","created":"2024-10-13","cited_by_count":7,"area":null},{"id":"2107.14324","title":"Deep Networks Provably Classify Data on Curves","created":"2021-07-29","cited_by_count":6,"area":null},{"id":"2109.14142","title":"On the Provable Generalization of Recurrent Neural Networks","created":"2021-09-29","cited_by_count":6,"area":null},{"id":"2110.07807","title":"Provable Regret Bounds for Deep Online Learning and Control","created":"2021-10-15","cited_by_count":6,"area":null},{"id":"2205.11716","title":"Randomly Initialized One-Layer Neural Networks Make Data Linearly Separable","created":"2022-05-24","cited_by_count":6,"area":null},{"id":"2206.02047","title":"On the Generalization Power of the Overfitted Three-Layer Neural Tangent Kernel Model","created":"2022-06-04","cited_by_count":6,"area":null},{"id":"2207.00411","title":"Adversarial Robustness is at Odds with Lazy Training","created":"2022-06-18","cited_by_count":6,"area":null},{"id":"2308.08358","title":"Convergence of Two-Layer Regression with Nonlinear Units","created":"2023-08-16","cited_by_count":6,"area":null},{"id":"2311.13624","title":"A Theoretical Insight into Attack and Defense of Gradient Leakage in Transformer","created":"2023-11-22","cited_by_count":6,"area":null},{"id":"2107.05341","title":"Nonparametric Regression with Shallow Overparameterized Neural Networks Trained by GD with Early Stopping","created":"2021-07-12","cited_by_count":5,"area":null},{"id":"2109.13419","title":"The Role of Lookahead and Approximate Policy Evaluation in Reinforcement Learning with Linear Value Function Approximation","created":"2021-09-28","cited_by_count":5,"area":null},{"id":"2203.08961","title":"On the Convergence of Certified Robust Training with Interval Bound Propagation","created":"2022-03-16","cited_by_count":5,"area":null},{"id":"2210.11452","title":"Global Convergence of SGD On Two Layer Neural Nets","created":"2022-10-20","cited_by_count":5,"area":null},{"id":"2212.13848","title":"Learning Lipschitz Functions by GD-trained Shallow Overparameterized ReLU Neural Networks","created":"2022-12-28","cited_by_count":5,"area":null},{"id":"2301.08117","title":"Convergence beyond the over-parameterized regime using Rayleigh quotients","created":"2023-01-19","cited_by_count":5,"area":null},{"id":"2502.17340","title":"Low-rank bias, weight decay, and model merging in neural networks","created":"2025-02-24","cited_by_count":5,"area":null},{"id":"2210.07338","title":"Reinforcement Learning with Unbiased Policy Evaluation and Linear Function Approximation","created":"2022-10-13","cited_by_count":4,"area":null},{"id":"2402.05696","title":"Fixed width treelike neural networks capacity analysis -- generic activations","created":"2024-02-08","cited_by_count":4,"area":null},{"id":"2410.09758","title":"BiDoRA: Bi-level Optimization-Based Weight-Decomposed Low-Rank Adaptation","created":"2024-10-13","cited_by_count":4,"area":null},{"id":"2104.11895","title":"Achieving Small Test Error in Mildly Overparameterized Neural Networks","created":"2021-04-24","cited_by_count":3,"area":null},{"id":"2210.00176","title":"A Combinatorial Perspective on the Optimization of Shallow ReLU Networks","created":"2022-10-01","cited_by_count":3,"area":null},{"id":"2301.00335","title":"Pruning Before Training May Improve Generalization, Provably","created":"2023-01-01","cited_by_count":3,"area":null},{"id":"2309.09258","title":"Global Convergence of SGD For Logistic Loss on Two Layer Neural Nets","created":"2023-09-17","cited_by_count":3,"area":null},{"id":"2312.08244","title":"Capacity of the treelike sign perceptrons neural networks with one hidden layer -- RDT based upper bounds","created":"2023-12-13","cited_by_count":3,"area":null},{"id":"2312.08257","title":"\\emph{Lifted} RDT based capacity analysis of the 1-hidden layer treelike \\emph{sign} perceptrons neural networks","created":"2023-12-13","cited_by_count":3,"area":null},{"id":"2412.14031","title":"A Riemannian Optimization Perspective of the Gauss-Newton Method for Feedforward Neural Networks","created":"2024-12-18","cited_by_count":3,"area":null},{"id":"2504.11130","title":"Divergence of Empirical Neural Tangent Kernel in Classification Problems","created":"2025-04-15","cited_by_count":3,"area":null},{"id":"2510.02779","title":"Optimal Rates for Generalization of Gradient Descent for Deep ReLU Classification","created":"2025-10-03","cited_by_count":3,"area":null},{"id":"2510.05573","title":"On the Theory of Continual Learning with Gradient Descent for Neural Networks","created":"2025-10-07","cited_by_count":3,"area":null},{"id":"2102.13522","title":"Experiments with Rich Regime Training for Deep Learning","created":"2021-02-26","cited_by_count":2,"area":null},{"id":"2106.04795","title":"Nonasymptotic theory for two-layer neural networks: Beyond the bias-variance trade-off","created":"2021-06-09","cited_by_count":2,"area":null},{"id":"2301.00327","title":"Neural Networks with Sparse Activation Induced by Large Bias: Tighter Analysis with Bias-Generalized NTK","created":"2023-01-01","cited_by_count":2,"area":null},{"id":"2305.19640","title":"Fine-grained Analysis of Non-parametric Estimation for Pairwise Learning","created":"2023-05-31","cited_by_count":2,"area":null},{"id":"2309.04860","title":"Approximation Results for Gradient Descent trained Neural Networks","created":"2023-09-09","cited_by_count":2,"area":null},{"id":"2201.12052","title":"Improved Overparametrization Bounds for Global Convergence of Stochastic Gradient Descent for Shallow Neural Networks","created":"2022-01-28","cited_by_count":1,"area":null},{"id":"2205.14553","title":"Long-Tailed Learning Requires Feature Learning","created":"2022-05-29","cited_by_count":1,"area":null},{"id":"2212.02323","title":"Improved Convergence Guarantees for Shallow Neural Networks","created":"2022-12-05","cited_by_count":1,"area":null},{"id":"2306.07544","title":"On Achieving Optimal Adversarial Test Error","created":"2023-06-13","cited_by_count":1,"area":null},{"id":"2309.15096","title":"Fixing the NTK: From Neural Network Linearizations to Exact Convex Programs","created":"2023-09-26","cited_by_count":1,"area":null}],"next_cursor":"100"}