{"total":175,"rows":[{"id":"2101.12699","title":"Exploring Deep Neural Networks via Layer-Peeled Model: Minority Collapse in Imbalanced Training","created":"2021-01-29","cited_by_count":166,"area":null},{"id":"2206.15144","title":"Neural Networks can Learn Representations with Gradient Descent","created":"2022-06-30","cited_by_count":161,"area":null},{"id":"2101.10588","title":"Generalization error of random features and kernel methods: hypercontractivity and kernel matrix concentration","created":"2021-01-26","cited_by_count":110,"area":null},{"id":"2305.16380","title":"Scan and Snap: Understanding Training Dynamics and Token Composition in 1-layer Transformer","created":"2023-05-25","cited_by_count":102,"area":null},{"id":"2102.13219","title":"Learning with invariances in random features and kernel models","created":"2021-02-25","cited_by_count":89,"area":null},{"id":"2106.15013","title":"Small random initialization is akin to spectral learning: Optimization and generalization guarantees for overparameterized low-rank matrix reconstruction","created":"2021-06-28","cited_by_count":78,"area":null},{"id":"2112.00029","title":"Pixelated Butterfly: Simple and Efficient Sparse training for Neural Network Models","created":"2021-11-30","cited_by_count":76,"area":null},{"id":"2209.14863","title":"Neural Networks Efficiently Learn Low-Dimensional Representations with SGD","created":"2022-09-29","cited_by_count":63,"area":null},{"id":"2112.07628","title":"Training Multi-Layer Over-Parametrized Neural Network in Subquadratic Time","created":"2021-12-14","cited_by_count":62,"area":null},{"id":"2306.13596","title":"Max-Margin Token Selection in Attention Mechanism","created":"2023-06-23","cited_by_count":58,"area":null},{"id":"2310.00535","title":"JoMA: Demystifying Multilayer Transformers via JOint Dynamics of MLP and Attention","created":"2023-10-01","cited_by_count":52,"area":null},{"id":"2212.07469","title":"Learning threshold neurons via the \"edge of stability\"","created":"2022-12-14","cited_by_count":51,"area":null},{"id":"2105.09513","title":"Deep Kronecker neural networks: A general framework for neural networks with adaptive activation functions","created":"2021-05-20","cited_by_count":49,"area":null},{"id":"2201.01212","title":"AutoBalance: Optimized Loss Functions for Imbalanced Data","created":"2022-01-04","cited_by_count":49,"area":null},{"id":"2102.02410","title":"A Local Convergence Theory for Mildly Over-Parameterized Two-Layer Neural Network","created":"2021-02-04","cited_by_count":47,"area":null},{"id":"2303.16504","title":"An Over-parameterized Exponential Regression","created":"2023-03-29","cited_by_count":45,"area":null},{"id":"2101.09612","title":"On the Proof of Global Convergence of Gradient Descent for Deep ReLU Networks with Linear Widths","created":"2021-01-24","cited_by_count":44,"area":null},{"id":"2307.11353","title":"What can a Single Attention Layer Learn? A Study Through the Random Features Lens","created":"2023-07-21","cited_by_count":40,"area":null},{"id":"2310.12680","title":"On the Optimization and Generalization of Multi-head Attention","created":"2023-10-19","cited_by_count":37,"area":null},{"id":"2205.10217","title":"Memorization and Optimization in Deep Neural Networks with Minimum Over-parameterization","created":"2022-05-20","cited_by_count":36,"area":null},{"id":"2106.05932","title":"Early-stopped neural networks are consistent","created":"2021-06-10","cited_by_count":34,"area":null},{"id":"2110.04622","title":"Does Preprocessing Help Training Over-parameterized Neural Networks?","created":"2021-10-09","cited_by_count":32,"area":null},{"id":"2309.01020","title":"On the training and generalization of deep operator networks","created":"2023-09-02","cited_by_count":31,"area":null},{"id":"2211.06506","title":"Spectral Evolution and Invariance in Linear-width Neural Networks","created":"2022-11-11","cited_by_count":30,"area":null},{"id":"2309.07418","title":"A Fast Optimization View: Reformulating Single Layer Attention in LLM Based on Tensor and SVM Trick, and Solving It in Matrix Multiplication Time","created":"2023-09-14","cited_by_count":30,"area":null},{"id":"2211.14227","title":"Bypass Exponential Time Preprocessing: Fast Neural Network Training via Weight-Data Correlation Preprocessing","created":"2022-11-25","cited_by_count":29,"area":null},{"id":"2312.15995","title":"Generalization in Kernel Regression Under Realistic Assumptions","created":"2023-12-26","cited_by_count":27,"area":null},{"id":"2107.12723","title":"Stability & Generalisation of Gradient Descent for Shallow Neural Networks without the Neural Tangent Kernel","created":"2021-07-27","cited_by_count":26,"area":null},{"id":"2111.01875","title":"Subquadratic Overparameterization for Shallow Neural Networks","created":"2021-11-02","cited_by_count":26,"area":null},{"id":"2308.12247","title":"How to Protect Copyright Data in Optimization of Large Language Models?","created":"2023-08-23","cited_by_count":26,"area":null},{"id":"2102.05320","title":"An Adaptive Stochastic Sequential Quadratic Programming with Differentiable Exact Augmented Lagrangians","created":"2021-02-10","cited_by_count":25,"area":null},{"id":"2302.10034","title":"Over-Parameterization Exponentially Slows Down Gradient Descent for Learning a Single Neuron","created":"2023-02-20","cited_by_count":25,"area":null},{"id":"2306.08321","title":"Nonparametric regression using over-parameterized shallow ReLU neural networks","created":"2023-06-14","cited_by_count":24,"area":null},{"id":"2203.09137","title":"Global Convergence of MAML and Theory-Inspired Neural Architecture Search for Few-Shot Learning","created":"2022-03-17","cited_by_count":22,"area":null},{"id":"2208.04508","title":"Training Overparametrized Neural Networks in Sublinear Time","created":"2022-08-09","cited_by_count":22,"area":null},{"id":"2208.05395","title":"A Sublinear Adversarial Training Algorithm","created":"2022-08-10","cited_by_count":22,"area":null},{"id":"2211.07844","title":"Characterizing the Spectrum of the NTK via a Power Series Expansion","created":"2022-11-15","cited_by_count":22,"area":null},{"id":"2109.09304","title":"Deformed semicircle law and concentration of nonlinear random matrices for ultra-wide neural networks","created":"2021-09-20","cited_by_count":21,"area":null},{"id":"2302.09235","title":"Generalization and Stability of Interpolating Neural Networks with Minimal Width","created":"2023-02-18","cited_by_count":21,"area":null},{"id":"2106.06573","title":"Understanding Deflation Process in Over-parametrized Tensor Decomposition","created":"2021-06-11","cited_by_count":20,"area":null},{"id":"2206.00833","title":"Finite-Time Analysis of Entropy-Regularized Neural Natural Actor-Critic Algorithm","created":"2022-06-02","cited_by_count":20,"area":null},{"id":"2308.13431","title":"Six Lectures on Linearized Neural Networks","created":"2023-08-25","cited_by_count":20,"area":null},{"id":"2401.07187","title":"A Survey on Statistical Theory of Deep Learning: Approximation, Training Dynamics, and Generative Models","created":"2024-01-14","cited_by_count":20,"area":null},{"id":"2209.07263","title":"Robustness in deep learning: The good (width), the bad (depth), and the ugly (initialization)","created":"2022-09-15","cited_by_count":18,"area":null},{"id":"2110.05645","title":"A global convergence theory for deep ReLU implicit networks via over-parameterization","created":"2021-10-11","cited_by_count":17,"area":null},{"id":"2305.19510","title":"Mildly Overparameterized ReLU Networks Have a Favorable Loss Landscape","created":"2023-05-31","cited_by_count":17,"area":null},{"id":"2406.03334","title":"Reparameterization invariance in approximate Bayesian inference","created":"2024-06-05","cited_by_count":17,"area":null},{"id":"2302.02922","title":"Joint Edge-Model Sparse Learning is Provably Efficient for Graph Neural Networks","created":"2023-02-06","cited_by_count":16,"area":null},{"id":"2307.06887","title":"Provable Multi-Task Representation Learning by Two-Layer ReLU Neural Networks","created":"2023-07-13","cited_by_count":16,"area":null},{"id":"2310.18123","title":"Sample Complexity Bounds for Score-Matching: Causal Discovery and Generative Modeling","created":"2023-10-27","cited_by_count":16,"area":null},{"id":"2112.05095","title":"Provable Continual Learning via Sketched Jacobian Approximations","created":"2021-12-09","cited_by_count":15,"area":null},{"id":"2206.12802","title":"Bounding the Width of Neural Networks via Coupled Initialization -- A Worst Case Analysis","created":"2022-06-26","cited_by_count":15,"area":null},{"id":"2104.05605","title":"Understanding Overparameterization in Generative Adversarial Networks","created":"2021-04-12","cited_by_count":14,"area":null},{"id":"2104.14132","title":"Generalization Guarantees for Neural Architecture Search with Train-Validation Split","created":"2021-04-29","cited_by_count":14,"area":null},{"id":"2110.05667","title":"Why Lottery Ticket Wins? A Theoretical Perspective of Sample Complexity on Pruned Neural Networks","created":"2021-10-12","cited_by_count":14,"area":null},{"id":"2209.07238","title":"Generalization Properties of NAS under Activation and Skip Connection Search","created":"2022-09-15","cited_by_count":14,"area":null},{"id":"2407.13621","title":"Differential Privacy Mechanisms in Neural Tangent Kernel Regression","created":"2024-07-18","cited_by_count":14,"area":null},{"id":"2105.05001","title":"FL-NTK: A Neural Tangent Kernel-based Framework for Federated Learning Convergence Analysis","created":"2021-05-11","cited_by_count":13,"area":null},{"id":"2204.10782","title":"On Feature Learning in Neural Networks with Global Convergence Guarantees","created":"2022-04-22","cited_by_count":13,"area":null},{"id":"2311.01575","title":"On the Convergence of Encoder-only Shallow Transformers","created":"2023-11-02","cited_by_count":13,"area":null},{"id":"2405.14630","title":"Bounds for the smallest eigenvalue of the NTK for arbitrary spherical data of arbitrary dimension","created":"2024-05-23","cited_by_count":13,"area":null},{"id":"2410.23169","title":"The Persistence of Neural Collapse Despite Low-Rank Bias","created":"2024-10-30","cited_by_count":13,"area":null},{"id":"2102.09671","title":"When Are Solutions Connected in Deep Networks?","created":"2021-02-18","cited_by_count":11,"area":null},{"id":"2105.06569","title":"The Dynamics of Gradient Descent for Overparametrized Neural Networks","created":"2021-05-13","cited_by_count":11,"area":null},{"id":"2112.02668","title":"On the Convergence of Shallow Neural Network Training with Randomly Masked Neurons","created":"2021-12-05","cited_by_count":11,"area":null},{"id":"2206.02927","title":"Spectral Bias Outside the Training Set for Deep Networks in the Kernel Regime","created":"2022-06-06","cited_by_count":11,"area":null},{"id":"2307.06565","title":"Efficient SGD Neural Network Training via Sublinear Activated Neuron Identification","created":"2023-07-13","cited_by_count":11,"area":null},{"id":"2403.19205","title":"From Activation to Initialization: Scaling Insights for Optimizing Neural Fields","created":"2024-03-28","cited_by_count":11,"area":null},{"id":"2103.01391","title":"Sample Complexity and Overparameterization Bounds for Temporal Difference Learning with Neural Network Approximation","created":"2021-03-02","cited_by_count":10,"area":null},{"id":"2205.13814","title":"Global Convergence of Over-parameterized Deep Equilibrium Models","created":"2022-05-27","cited_by_count":10,"area":null},{"id":"2201.04738","title":"Implicit Bias of MSE Gradient Optimization in Underparameterized Neural Networks","created":"2022-01-12","cited_by_count":9,"area":null},{"id":"2307.14531","title":"Controlling the Inductive Bias of Wide Neural Networks by Modifying the Kernel's Spectrum","created":"2023-07-26","cited_by_count":9,"area":null},{"id":"2310.11291","title":"An Automatic Learning Rate Schedule Algorithm for Achieving Faster Convergence and Steeper Descent","created":"2023-10-17","cited_by_count":9,"area":null},{"id":"2101.07150","title":"Stable Recovery of Entangled Weights: Towards Robust Identification of Deep Neural Networks from Minimal Samples","created":"2021-01-18","cited_by_count":8,"area":null},{"id":"2202.12262","title":"On the Omnipresence of Spurious Local Minima in Certain Neural Network Training Problems","created":"2022-02-23","cited_by_count":8,"area":null},{"id":"2210.12001","title":"When Expressivity Meets Trainability: Fewer than $n$ Neurons Can Work","created":"2022-10-21","cited_by_count":8,"area":null},{"id":"2303.14844","title":"Analyzing Convergence in Quantum Neural Networks: Deviations from Neural Tangent Kernels","created":"2023-03-26","cited_by_count":8,"area":null},{"id":"2309.07452","title":"Is Solving Graph Neural Tangent Kernel Equivalent to Training Graph Neural Network?","created":"2023-09-14","cited_by_count":8,"area":null},{"id":"2311.01644","title":"Should Under-parameterized Student Networks Copy or Average Teacher Weights?","created":"2023-11-03","cited_by_count":8,"area":null},{"id":"2402.12241","title":"Convergence of Gradient Descent for Recurrent Neural Networks: A Nonasymptotic Analysis","created":"2024-02-19","cited_by_count":8,"area":null},{"id":"2106.14749","title":"A Theory-Driven Self-Labeling Refinement Method for Contrastive Representation Learning","created":"2021-06-28","cited_by_count":7,"area":null},{"id":"2203.03445","title":"S-Rocket: Selective Random Convolution Kernels for Time Series Classification","created":"2022-03-07","cited_by_count":7,"area":null},{"id":"2209.08399","title":"Approximation results for Gradient Descent trained Shallow Neural Networks in $1d$","created":"2022-09-17","cited_by_count":7,"area":null},{"id":"2209.15106","title":"Restricted Strong Convexity of Deep Learning Models with Smooth Activations","created":"2022-09-29","cited_by_count":7,"area":null},{"id":"2210.05371","title":"On skip connections and normalisation layers in deep optimisation","created":"2022-10-10","cited_by_count":7,"area":null},{"id":"2211.06077","title":"Overparameterized random feature regression with nearly orthogonal data","created":"2022-11-11","cited_by_count":7,"area":null},{"id":"2203.05104","title":"Transition to Linearity of Wide Neural Networks is an Emerging Property of Assembling Weak Models","created":"2022-03-10","cited_by_count":6,"area":null},{"id":"2205.11716","title":"Randomly Initialized One-Layer Neural Networks Make Data Linearly Separable","created":"2022-05-24","cited_by_count":6,"area":null},{"id":"2305.08813","title":"Better NTK Conditioning: A Free Lunch from (ReLU) Nonlinear Activation in Wide Neural Networks","created":"2023-05-15","cited_by_count":6,"area":null},{"id":"2308.08358","title":"Convergence of Two-Layer Regression with Nonlinear Units","created":"2023-08-16","cited_by_count":6,"area":null},{"id":"2103.15996","title":"Minimum complexity interpolation in random features models","created":"2021-03-30","cited_by_count":5,"area":null},{"id":"2107.05341","title":"Nonparametric Regression with Shallow Overparameterized Neural Networks Trained by GD with Early Stopping","created":"2021-07-12","cited_by_count":5,"area":null},{"id":"2205.07463","title":"Gradient Descent Optimizes Infinite-Depth ReLU Implicit Networks with Linear Widths","created":"2022-05-16","cited_by_count":5,"area":null},{"id":"2211.04589","title":"Finite Sample Identification of Wide Shallow Neural Networks with Biases","created":"2022-11-08","cited_by_count":5,"area":null},{"id":"2212.13848","title":"Learning Lipschitz Functions by GD-trained Shallow Overparameterized ReLU Neural Networks","created":"2022-12-28","cited_by_count":5,"area":null},{"id":"2302.07828","title":"Over-parametrization via Lifting for Low-rank Matrix Sensing: Conversion of Spurious Solutions to Strict Saddle Points","created":"2023-02-15","cited_by_count":5,"area":null},{"id":"2307.01177","title":"Neural Hilbert Ladders: Multi-Layer Neural Networks in Function Space","created":"2023-07-03","cited_by_count":5,"area":null},{"id":"2308.02001","title":"Memory capacity of two layer neural networks with smooth activations","created":"2023-08-03","cited_by_count":5,"area":null},{"id":"2407.12354","title":"Invertible Neural Warp for NeRF","created":"2024-07-17","cited_by_count":5,"area":null},{"id":"2101.04243","title":"A Convergence Theory Towards Practical Over-parameterized Deep Neural Networks","created":"2021-01-12","cited_by_count":4,"area":null}],"next_cursor":"100"}