{"total":261,"rows":[{"id":"2106.09685","title":"LoRA: Low-Rank Adaptation of Large Language Models","created":"2021-06-17","cited_by_count":14380,"area":null},{"id":"2103.00065","title":"Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability","created":"2021-02-26","cited_by_count":427,"area":null},{"id":"2106.15590","title":"The Values Encoded in Machine Learning Research","created":"2021-06-29","cited_by_count":172,"area":null},{"id":"2206.15144","title":"Neural Networks can Learn Representations with Gradient Descent","created":"2022-06-30","cited_by_count":161,"area":null},{"id":"2210.05643","title":"A Kernel-Based View of Language Model Fine-Tuning","created":"2022-10-11","cited_by_count":121,"area":null},{"id":"2105.15134","title":"Toward Understanding the Feature Learning Process of Self-supervised Contrastive Learning","created":"2021-05-31","cited_by_count":119,"area":null},{"id":"2101.10588","title":"Generalization error of random features and kernel methods: hypercontractivity and kernel matrix concentration","created":"2021-01-26","cited_by_count":110,"area":null},{"id":"2102.02887","title":"Do We Actually Need Dense Over-Parameterization? In-Time Over-Parameterization in Sparse Training","created":"2021-02-04","cited_by_count":110,"area":null},{"id":"2110.06914","title":"What Happens after SGD Reaches Zero Loss? --A Mathematical Framework","created":"2021-10-13","cited_by_count":107,"area":null},{"id":"2305.16380","title":"Scan and Snap: Understanding Training Dynamics and Token Composition in 1-layer Transformer","created":"2023-05-25","cited_by_count":102,"area":null},{"id":"2202.06526","title":"Benign Overfitting in Two-layer Convolutional Neural Networks","created":"2022-02-14","cited_by_count":91,"area":null},{"id":"2102.13219","title":"Learning with invariances in random features and kernel models","created":"2021-02-25","cited_by_count":89,"area":null},{"id":"2204.01050","title":"Understanding the unstable convergence of gradient descent","created":"2022-04-03","cited_by_count":86,"area":null},{"id":"2206.07085","title":"Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction","created":"2022-06-14","cited_by_count":86,"area":null},{"id":"2302.06015","title":"A Theoretical Understanding of Shallow Vision Transformers: Learning, Generalization, and Sample Complexity","created":"2023-02-12","cited_by_count":77,"area":null},{"id":"2112.00029","title":"Pixelated Butterfly: Simple and Efficient Sparse training for Neural Network Models","created":"2021-11-30","cited_by_count":76,"area":null},{"id":"2308.16898","title":"Transformers as Support Vector Machines","created":"2023-08-31","cited_by_count":74,"area":null},{"id":"2105.14602","title":"On the geometry of generalization and memorization in deep neural networks","created":"2021-05-30","cited_by_count":73,"area":null},{"id":"2102.11742","title":"Classifying high-dimensional Gaussian mixtures: Where kernel methods fail and neural networks succeed","created":"2021-02-23","cited_by_count":69,"area":null},{"id":"2112.07628","title":"Training Multi-Layer Over-Parametrized Neural Network in Subquadratic Time","created":"2021-12-14","cited_by_count":62,"area":null},{"id":"2105.04550","title":"Optimization of Graph Neural Networks: Implicit Acceleration by Skip Connections and More Depth","created":"2021-05-10","cited_by_count":55,"area":null},{"id":"2208.02813","title":"Towards Understanding Mixture of Experts in Deep Learning","created":"2022-08-04","cited_by_count":52,"area":"moe"},{"id":"2310.00535","title":"JoMA: Demystifying Multilayer Transformers via JOint Dynamics of MLP and Attention","created":"2023-10-01","cited_by_count":52,"area":null},{"id":"2108.11371","title":"Understanding the Generalization of Adam in Learning Neural Networks with Proper Regularization","created":"2021-08-25","cited_by_count":49,"area":null},{"id":"2102.02410","title":"A Local Convergence Theory for Mildly Over-Parameterized Two-Layer Neural Network","created":"2021-02-04","cited_by_count":47,"area":null},{"id":"2204.10425","title":"Spectrum of inner-product kernel matrices in the polynomial regime and multiple descent phenomenon in kernel ridge regression","created":"2022-04-21","cited_by_count":45,"area":null},{"id":"2303.16504","title":"An Over-parameterized Exponential Regression","created":"2023-03-29","cited_by_count":45,"area":null},{"id":"2103.01210","title":"Quantifying the Benefit of Using Differentiable Learning over Tangent Kernels","created":"2021-03-01","cited_by_count":41,"area":null},{"id":"2307.11353","title":"What can a Single Attention Layer Learn? A Study Through the Random Features Lens","created":"2023-07-21","cited_by_count":40,"area":null},{"id":"2209.13083","title":"Why neural networks find simple solutions: the many regularizers of geometric complexity","created":"2022-09-27","cited_by_count":39,"area":null},{"id":"2302.01107","title":"A Survey on Efficient Training of Transformers","created":"2023-02-02","cited_by_count":39,"area":null},{"id":"2102.02515","title":"HYDRA: Hypergradient Data Relevance Analysis for Interpreting Deep Neural Networks","created":"2021-02-04","cited_by_count":38,"area":null},{"id":"2102.07346","title":"On the Theory of Implicit Deep Learning: Global Convergence with Implicit Layers","created":"2021-02-15","cited_by_count":35,"area":null},{"id":"2306.04073","title":"Patch-level Routing in Mixture-of-Experts is Provably Sample-efficient for Convolutional Neural Networks","created":"2023-06-07","cited_by_count":35,"area":"moe"},{"id":"2106.05932","title":"Early-stopped neural networks are consistent","created":"2021-06-10","cited_by_count":34,"area":null},{"id":"2107.00595","title":"Fast Margin Maximization via Dual Acceleration","created":"2021-07-01","cited_by_count":33,"area":null},{"id":"2108.09420","title":"Fast Sketching of Polynomial Kernels of Polynomial Degree","created":"2021-08-21","cited_by_count":33,"area":null},{"id":"2110.04622","title":"Does Preprocessing Help Training Over-parameterized Neural Networks?","created":"2021-10-09","cited_by_count":32,"area":null},{"id":"2201.07705","title":"GEMEL: Model Merging for Memory-Efficient, Real-Time Video Analytics at the Edge","created":"2022-01-19","cited_by_count":32,"area":null},{"id":"2305.02657","title":"On the Eigenvalue Decay Rates of a Class of Neural-Network Related Kernel Functions Defined on General Domains","created":"2023-05-04","cited_by_count":31,"area":null},{"id":"2106.09913","title":"Iterative Feature Matching: Toward Provable Domain Generalization with Logarithmic Environments","created":"2021-06-18","cited_by_count":30,"area":null},{"id":"2309.07418","title":"A Fast Optimization View: Reformulating Single Layer Attention in LLM Based on Tensor and SVM Trick, and Solving It in Matrix Multiplication Time","created":"2023-09-14","cited_by_count":30,"area":null},{"id":"2205.12141","title":"One-Pixel Shortcut: on the Learning Preference of Deep Neural Networks","created":"2022-05-24","cited_by_count":29,"area":null},{"id":"2211.14227","title":"Bypass Exponential Time Preprocessing: Fast Neural Network Training via Weight-Data Correlation Preprocessing","created":"2022-11-25","cited_by_count":29,"area":null},{"id":"2105.10585","title":"Properties of the After Kernel","created":"2021-05-21","cited_by_count":27,"area":null},{"id":"2111.01875","title":"Subquadratic Overparameterization for Shallow Neural Networks","created":"2021-11-02","cited_by_count":26,"area":null},{"id":"2112.08297","title":"Rethinking Influence Functions of Neural Networks in the Over-parameterized Regime","created":"2021-12-15","cited_by_count":26,"area":null},{"id":"2308.12247","title":"How to Protect Copyright Data in Optimization of Large Language Models?","created":"2023-08-23","cited_by_count":26,"area":null},{"id":"2406.12839","title":"Evaluating the design space of diffusion-based generative models","created":"2024-06-18","cited_by_count":26,"area":"diffusion"},{"id":"2203.10036","title":"On the Generalization Mystery in Deep Learning","created":"2022-03-18","cited_by_count":25,"area":null},{"id":"2302.10034","title":"Over-Parameterization Exponentially Slows Down Gradient Descent for Learning a Single Neuron","created":"2023-02-20","cited_by_count":25,"area":null},{"id":"2105.14417","title":"Overparameterization of deep ResNet: zero loss and mean-field analysis","created":"2021-05-30","cited_by_count":24,"area":null},{"id":"2203.16462","title":"Convergence of gradient descent for deep neural networks","created":"2022-03-30","cited_by_count":24,"area":null},{"id":"2207.02099","title":"An Empirical Study of Implicit Regularization in Deep Offline RL","created":"2022-07-05","cited_by_count":24,"area":"offlinerl"},{"id":"2103.03191","title":"Generalization Bounds for Sparse Random Feature Expansions","created":"2021-03-04","cited_by_count":22,"area":null},{"id":"2208.04508","title":"Training Overparametrized Neural Networks in Sublinear Time","created":"2022-08-09","cited_by_count":22,"area":null},{"id":"2208.05395","title":"A Sublinear Adversarial Training Algorithm","created":"2022-08-10","cited_by_count":22,"area":null},{"id":"2302.09235","title":"Generalization and Stability of Interpolating Neural Networks with Minimal Width","created":"2023-02-18","cited_by_count":21,"area":null},{"id":"2405.16646","title":"A Provably Effective Method for Pruning Experts in Fine-tuned Sparse Mixture-of-Experts","created":"2024-05-26","cited_by_count":21,"area":"moe"},{"id":"2105.00303","title":"RATT: Leveraging Unlabeled Data to Guarantee Generalization","created":"2021-05-01","cited_by_count":20,"area":null},{"id":"2106.06573","title":"Understanding Deflation Process in Over-parametrized Tensor Decomposition","created":"2021-06-11","cited_by_count":20,"area":null},{"id":"2308.13431","title":"Six Lectures on Linearized Neural Networks","created":"2023-08-25","cited_by_count":20,"area":null},{"id":"2401.07187","title":"A Survey on Statistical Theory of Deep Learning: Approximation, Training Dynamics, and Generative Models","created":"2024-01-14","cited_by_count":20,"area":null},{"id":"2101.02533","title":"Towards Understanding Learning in Neural Networks with Linear Teachers","created":"2021-01-07","cited_by_count":19,"area":null},{"id":"2201.08514","title":"How does unlabeled data improve generalization in self-training? A one-hidden-layer theoretical analysis","created":"2022-01-21","cited_by_count":19,"area":null},{"id":"2110.05645","title":"A global convergence theory for deep ReLU implicit networks via over-parameterization","created":"2021-10-11","cited_by_count":17,"area":null},{"id":"2208.03635","title":"Federated Adversarial Learning: A Framework with Convergence Analysis","created":"2022-08-07","cited_by_count":17,"area":null},{"id":"2310.16173","title":"On the Convergence and Sample Complexity Analysis of Deep Q-Networks with $\\epsilon$-Greedy Exploration","created":"2023-10-24","cited_by_count":17,"area":null},{"id":"2502.00896","title":"LoR-VP: Low-Rank Visual Prompting for Efficient Vision Model Adaptation","created":"2025-02-02","cited_by_count":17,"area":null},{"id":"2212.14852","title":"An Analysis of Attention via the Lens of Exchangeability and Latent Variable Models","created":"2022-12-30","cited_by_count":16,"area":null},{"id":"2302.02922","title":"Joint Edge-Model Sparse Learning is Provably Efficient for Graph Neural Networks","created":"2023-02-06","cited_by_count":16,"area":null},{"id":"2307.06887","title":"Provable Multi-Task Representation Learning by Two-Layer ReLU Neural Networks","created":"2023-07-13","cited_by_count":16,"area":null},{"id":"2106.03227","title":"Neural Tangent Kernel Maximum Mean Discrepancy","created":"2021-06-06","cited_by_count":15,"area":null},{"id":"2107.04518","title":"Optimal Gradient-based Algorithms for Non-concave Bandit Optimization","created":"2021-07-09","cited_by_count":15,"area":null},{"id":"2206.12802","title":"Bounding the Width of Neural Networks via Coupled Initialization -- A Worst Case Analysis","created":"2022-06-26","cited_by_count":15,"area":null},{"id":"2310.11685","title":"Why Softmax Attention Outperforms Linear Attention","created":"2023-10-18","cited_by_count":15,"area":null},{"id":"2102.09924","title":"A proof of convergence for gradient descent in the training of artificial neural networks for constant target functions","created":"2021-02-19","cited_by_count":14,"area":null},{"id":"2108.00207","title":"The Separation Capacity of Random Neural Networks","created":"2021-07-31","cited_by_count":14,"area":null},{"id":"2112.03657","title":"Understanding Square Loss in Training Overparametrized Neural Network Classifiers","created":"2021-12-07","cited_by_count":14,"area":null},{"id":"2208.02789","title":"Feature selection with gradient descent on two-layer networks in low-rotation regimes","created":"2022-08-04","cited_by_count":14,"area":null},{"id":"2407.13621","title":"Differential Privacy Mechanisms in Neural Tangent Kernel Regression","created":"2024-07-18","cited_by_count":14,"area":null},{"id":"2101.07367","title":"Training Learned Optimizers with Randomly Initialized Learned Optimizers","created":"2021-01-14","cited_by_count":13,"area":null},{"id":"2103.05243","title":"On the Generalization Power of Overfitted Two-Layer Neural Tangent Kernel Models","created":"2021-03-09","cited_by_count":13,"area":null},{"id":"2105.05001","title":"FL-NTK: A Neural Tangent Kernel-based Framework for Federated Learning Convergence Analysis","created":"2021-05-11","cited_by_count":13,"area":null},{"id":"2204.10782","title":"On Feature Learning in Neural Networks with Global Convergence Guarantees","created":"2022-04-22","cited_by_count":13,"area":null},{"id":"2206.03688","title":"Identifying good directions to escape the NTK regime and efficiently learn low-degree plus sparse polynomials","created":"2022-06-08","cited_by_count":13,"area":null},{"id":"2307.05635","title":"Fundamental limits of overparametrized shallow neural networks for supervised learning","created":"2023-07-11","cited_by_count":13,"area":null},{"id":"2311.01575","title":"On the Convergence of Encoder-only Shallow Transformers","created":"2023-11-02","cited_by_count":13,"area":null},{"id":"2405.14630","title":"Bounds for the smallest eigenvalue of the NTK for arbitrary spherical data of arbitrary dimension","created":"2024-05-23","cited_by_count":13,"area":null},{"id":"2405.15018","title":"What Variables Affect Out-of-Distribution Generalization in Pretrained Models?","created":"2024-05-23","cited_by_count":13,"area":null},{"id":"2101.06309","title":"Fundamental Tradeoffs in Distributionally Adversarial Training","created":"2021-01-15","cited_by_count":12,"area":null},{"id":"2110.02926","title":"On the Global Convergence of Gradient Descent for multi-layer ResNets in the mean-field regime","created":"2021-10-06","cited_by_count":12,"area":null},{"id":"2110.04328","title":"Distinguishing rule- and exemplar-based generalization in learning systems","created":"2021-10-08","cited_by_count":12,"area":null},{"id":"2202.06438","title":"Learning from Randomly Initialized Neural Network Features","created":"2022-02-13","cited_by_count":12,"area":null},{"id":"2205.08099","title":"Dimensionality Reduced Training by Pruning and Freezing Parts of a Deep Neural Network, a Survey","created":"2022-05-17","cited_by_count":12,"area":null},{"id":"2207.13853","title":"ORFit: One-Pass Learning via Bridging Orthogonal Gradient Descent and Recursive Least-Squares","created":"2022-07-28","cited_by_count":12,"area":null},{"id":"2403.06398","title":"On the Diminishing Returns of Width for Continual Learning","created":"2024-03-11","cited_by_count":12,"area":null},{"id":"2404.08602","title":"Sliding down the stairs: how correlated latent variables accelerate learning with neural networks","created":"2024-04-12","cited_by_count":12,"area":null},{"id":"2405.16601","title":"A CMDP-within-online framework for Meta-Safe Reinforcement Learning","created":"2024-05-26","cited_by_count":12,"area":"deeprl"},{"id":"2101.07833","title":"Implicit Bias of Linear RNNs","created":"2021-01-19","cited_by_count":11,"area":null}],"next_cursor":"100"}