{"total":787,"rows":[{"id":"2106.09685","title":"LoRA: Low-Rank Adaptation of Large Language Models","created":"2021-06-17","cited_by_count":14380,"area":null},{"id":"2103.10385","title":"GPT Understands, Too","created":"2021-03-18","cited_by_count":716,"area":null},{"id":"2102.07623","title":"FedBN: Federated Learning on Non-IID Features via Local Batch Normalization","created":"2021-02-15","cited_by_count":571,"area":null},{"id":"2203.02053","title":"Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learning","created":"2022-03-03","cited_by_count":561,"area":null},{"id":"2103.00065","title":"Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability","created":"2021-02-26","cited_by_count":427,"area":null},{"id":"2105.02375","title":"A Geometric Analysis of Neural Collapse with Unconstrained Features","created":"2021-05-06","cited_by_count":227,"area":null},{"id":"2307.05695","title":"ReLoRA: High-Rank Training Through Low-Rank Updates","created":"2023-07-11","cited_by_count":204,"area":null},{"id":"2107.00651","title":"AutoFormer: Searching Transformers for Visual Recognition","created":"2021-07-01","cited_by_count":188,"area":null},{"id":"2207.08799","title":"Hidden Progress in Deep Learning: SGD Learns Parities Near the Computational Limit","created":"2022-07-18","cited_by_count":187,"area":null},{"id":"2106.15590","title":"The Values Encoded in Machine Learning Research","created":"2021-06-29","cited_by_count":172,"area":null},{"id":"2101.12699","title":"Exploring Deep Neural Networks via Layer-Peeled Model: Minority Collapse in Imbalanced Training","created":"2021-01-29","cited_by_count":166,"area":null},{"id":"2206.15144","title":"Neural Networks can Learn Representations with Gradient Descent","created":"2022-06-30","cited_by_count":161,"area":null},{"id":"2109.00650","title":"Dash: Semi-Supervised Learning with Dynamic Thresholding","created":"2021-09-01","cited_by_count":148,"area":null},{"id":"2206.03126","title":"Signal Propagation in Transformers: Theoretical Perspectives and the Role of Rank Collapse","created":"2022-06-07","cited_by_count":141,"area":null},{"id":"2305.18703","title":"Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Survey","created":"2023-05-30","cited_by_count":135,"area":null},{"id":"2112.08304","title":"On the Convergence and Robustness of Adversarial Training","created":"2021-12-15","cited_by_count":130,"area":null},{"id":"2205.09745","title":"Understanding Gradient Descent on Edge of Stability in Deep Learning","created":"2022-05-19","cited_by_count":128,"area":null},{"id":"2210.05643","title":"A Kernel-Based View of Language Model Fine-Tuning","created":"2022-10-11","cited_by_count":121,"area":null},{"id":"2105.15134","title":"Toward Understanding the Feature Learning Process of Self-supervised Contrastive Learning","created":"2021-05-31","cited_by_count":119,"area":null},{"id":"2101.10588","title":"Generalization error of random features and kernel methods: hypercontractivity and kernel matrix concentration","created":"2021-01-26","cited_by_count":110,"area":null},{"id":"2102.02887","title":"Do We Actually Need Dense Over-Parameterization? In-Time Over-Parameterization in Sparse Training","created":"2021-02-04","cited_by_count":110,"area":null},{"id":"2407.20311","title":"Physics of Language Models: Part 2.1, Grade-School Math and the Hidden Reasoning Process","created":"2024-07-29","cited_by_count":108,"area":"lr"},{"id":"2110.06914","title":"What Happens after SGD Reaches Zero Loss? --A Mathematical Framework","created":"2021-10-13","cited_by_count":107,"area":null},{"id":"2205.09653","title":"Self-Consistent Dynamical Field Theory of Kernel Evolution in Wide Neural Networks","created":"2022-05-19","cited_by_count":105,"area":null},{"id":"2305.16380","title":"Scan and Snap: Understanding Training Dynamics and Token Composition in 1-layer Transformer","created":"2023-05-25","cited_by_count":102,"area":null},{"id":"2109.11676","title":"Theory of overparametrization in quantum neural networks","created":"2021-09-23","cited_by_count":100,"area":null},{"id":"2206.04817","title":"The Slingshot Mechanism: An Empirical Study of Adaptive Optimizers and the Grokking Phenomenon","created":"2022-06-10","cited_by_count":94,"area":null},{"id":"2202.06526","title":"Benign Overfitting in Two-layer Convolutional Neural Networks","created":"2022-02-14","cited_by_count":91,"area":null},{"id":"2207.00713","title":"q-Learning in Continuous Time","created":"2022-07-02","cited_by_count":91,"area":null},{"id":"2110.01654","title":"Improved architectures and training algorithms for deep operator networks","created":"2021-10-04","cited_by_count":86,"area":null},{"id":"2206.07085","title":"Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction","created":"2022-06-14","cited_by_count":86,"area":null},{"id":"2202.02643","title":"The Unreasonable Effectiveness of Random Pruning: Return of the Most Naive Baseline for Sparse Training","created":"2022-02-05","cited_by_count":85,"area":null},{"id":"2102.00931","title":"Information-Theoretic Generalization Bounds for Stochastic Gradient Descent","created":"2021-02-01","cited_by_count":84,"area":null},{"id":"2110.13905","title":"Gradient Descent on Two-layer Nets: Margin Maximization and Simplicity Bias","created":"2021-10-26","cited_by_count":83,"area":null},{"id":"2305.13673","title":"Physics of Language Models: Part 1, Learning Hierarchical Language Structures","created":"2023-05-23","cited_by_count":83,"area":null},{"id":"2311.18817","title":"Dichotomy of Early and Late Phase Implicit Biases Can Provably Induce Grokking","created":"2023-11-30","cited_by_count":80,"area":null},{"id":"2202.07371","title":"Personalized Prompt Learning for Explainable Recommendation","created":"2022-02-15","cited_by_count":79,"area":null},{"id":"2302.06015","title":"A Theoretical Understanding of Shallow Vision Transformers: Learning, Generalization, and Sample Complexity","created":"2023-02-12","cited_by_count":77,"area":null},{"id":"2112.00029","title":"Pixelated Butterfly: Simple and Efficient Sparse training for Neural Network Models","created":"2021-11-30","cited_by_count":76,"area":null},{"id":"2312.14688","title":"A Mathematical Guide to Operator Learning","created":"2023-12-22","cited_by_count":76,"area":null},{"id":"2202.05928","title":"Benign Overfitting without Linearity: Neural Network Classifiers Trained by Gradient Descent for Noisy Linear Data","created":"2022-02-11","cited_by_count":74,"area":null},{"id":"2206.00187","title":"DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Training","created":"2022-06-01","cited_by_count":74,"area":null},{"id":"2308.16898","title":"Transformers as Support Vector Machines","created":"2023-08-31","cited_by_count":74,"area":null},{"id":"2102.12470","title":"On the Validity of Modeling SGD with Stochastic Differential Equations (SDEs)","created":"2021-02-24","cited_by_count":73,"area":null},{"id":"2102.11742","title":"Classifying high-dimensional Gaussian mixtures: Where kernel methods fail and neural networks succeed","created":"2021-02-23","cited_by_count":69,"area":null},{"id":"2201.04122","title":"In Defense of the Unitary Scalarization for Deep Multi-Task Learning","created":"2022-01-11","cited_by_count":67,"area":null},{"id":"2105.11144","title":"Improved OOD Generalization via Adversarial Training and Pre-training","created":"2021-05-24","cited_by_count":66,"area":null},{"id":"2305.03937","title":"Residual Prompt Tuning: Improving Prompt Tuning with Residual Reparameterization","created":"2023-05-06","cited_by_count":64,"area":null},{"id":"2209.14863","title":"Neural Networks Efficiently Learn Low-Dimensional Representations with SGD","created":"2022-09-29","cited_by_count":63,"area":null},{"id":"2112.07628","title":"Training Multi-Layer Over-Parametrized Neural Network in Subquadratic Time","created":"2021-12-14","cited_by_count":62,"area":null},{"id":"2106.12379","title":"AC/DC: Alternating Compressed/DeCompressed Training of Deep Neural Networks","created":"2021-06-23","cited_by_count":59,"area":null},{"id":"2301.11300","title":"ZiCo: Zero-shot NAS via Inverse Coefficient of Variation on Gradients","created":"2023-01-26","cited_by_count":58,"area":null},{"id":"2302.10911","title":"Revisiting Weighted Aggregation in Federated Learning with Neural Networks","created":"2023-02-14","cited_by_count":58,"area":null},{"id":"2306.13596","title":"Max-Margin Token Selection in Attention Mechanism","created":"2023-06-23","cited_by_count":58,"area":null},{"id":"2105.03703","title":"Tensor Programs IIb: Architectural Universality of Neural Tangent Kernel Training Dynamics","created":"2021-05-08","cited_by_count":57,"area":null},{"id":"2303.03108","title":"Gradient Norm Aware Minimization Seeks First-Order Flatness and Improves Generalization","created":"2023-03-03","cited_by_count":57,"area":null},{"id":"2105.04550","title":"Optimization of Graph Neural Networks: Implicit Acceleration by Skip Connections and More Depth","created":"2021-05-10","cited_by_count":55,"area":null},{"id":"2106.01606","title":"Exploring Memorization in Adversarial Training","created":"2021-06-03","cited_by_count":55,"area":null},{"id":"2112.15383","title":"Separation of Scales and a Thermodynamic Description of Feature Learning in Some CNNs","created":"2021-12-31","cited_by_count":53,"area":null},{"id":"2208.02813","title":"Towards Understanding Mixture of Experts in Deep Learning","created":"2022-08-04","cited_by_count":52,"area":"moe"},{"id":"2210.09974","title":"Theoretical Guarantees for Permutation-Equivariant Quantum Neural Networks","created":"2022-10-18","cited_by_count":52,"area":null},{"id":"2310.00535","title":"JoMA: Demystifying Multilayer Transformers via JOint Dynamics of MLP and Attention","created":"2023-10-01","cited_by_count":52,"area":null},{"id":"2212.07469","title":"Learning threshold neurons via the \"edge of stability\"","created":"2022-12-14","cited_by_count":51,"area":null},{"id":"2106.09017","title":"Bridging Multi-Task Learning and Meta-Learning: Towards Efficient Training and Effective Adaptation","created":"2021-06-16","cited_by_count":50,"area":null},{"id":"2105.09513","title":"Deep Kronecker neural networks: A general framework for neural networks with adaptive activation functions","created":"2021-05-20","cited_by_count":49,"area":null},{"id":"2108.11371","title":"Understanding the Generalization of Adam in Learning Neural Networks with Proper Regularization","created":"2021-08-25","cited_by_count":49,"area":null},{"id":"2206.01717","title":"A Theoretical Analysis on Feature Learning in Neural Networks: Emergence from Inputs and Advantage over Fixed Features","created":"2022-06-03","cited_by_count":49,"area":null},{"id":"2304.10411","title":"Attention Scheme Inspired Softmax Regression","created":"2023-04-20","cited_by_count":49,"area":null},{"id":"2409.14623","title":"From Lazy to Rich: Exact Learning Dynamics in Deep Linear Networks","created":"2024-09-22","cited_by_count":49,"area":null},{"id":"2207.10276","title":"ProMix: Combating Label Noise via Maximizing Clean Sample Utility","created":"2022-07-21","cited_by_count":48,"area":null},{"id":"2402.11867","title":"LoRA Training in the NTK Regime has No Spurious Local Minima","created":"2024-02-19","cited_by_count":48,"area":null},{"id":"2102.02410","title":"A Local Convergence Theory for Mildly Over-Parameterized Two-Layer Neural Network","created":"2021-02-04","cited_by_count":47,"area":null},{"id":"2210.07082","title":"Implicit Bias in Leaky ReLU Networks Trained on High-Dimensional Data","created":"2022-10-13","cited_by_count":46,"area":null},{"id":"2303.16504","title":"An Over-parameterized Exponential Regression","created":"2023-03-29","cited_by_count":45,"area":null},{"id":"2101.09612","title":"On the Proof of Global Convergence of Gradient Descent for Deep ReLU Networks with Linear Widths","created":"2021-01-24","cited_by_count":44,"area":null},{"id":"2105.06072","title":"Leveraging Non-uniformity in First-order Non-convex Optimization","created":"2021-05-13","cited_by_count":44,"area":null},{"id":"2110.03177","title":"EE-Net: Exploitation-Exploration Neural Networks in Contextual Bandits","created":"2021-10-07","cited_by_count":44,"area":null},{"id":"2107.14702","title":"Towards General Function Approximation in Zero-Sum Markov Games","created":"2021-07-30","cited_by_count":43,"area":null},{"id":"2204.11326","title":"Beyond the Quadratic Approximation: the Multiscale Structure of Neural Network Loss Landscapes","created":"2022-04-24","cited_by_count":43,"area":null},{"id":"2401.15604","title":"Neural Network-Based Score Estimation in Diffusion Models: Optimization and Generalization","created":"2024-01-28","cited_by_count":42,"area":"diffusion"},{"id":"2406.06158","title":"Get rich quick: exact solutions reveal how unbalanced initializations promote rapid feature learning","created":"2024-06-10","cited_by_count":42,"area":null},{"id":"2206.02768","title":"The Neural Covariance SDE: Shaped Infinite Depth-and-Width Networks at Initialization","created":"2022-06-06","cited_by_count":41,"area":null},{"id":"2307.11353","title":"What can a Single Attention Layer Learn? A Study Through the Random Features Lens","created":"2023-07-21","cited_by_count":40,"area":null},{"id":"2102.05375","title":"Strength of Minibatch Noise in SGD","created":"2021-02-10","cited_by_count":39,"area":null},{"id":"2207.04036","title":"Implicit Bias of Gradient Descent on Reparametrized Models: On Equivalence to Mirror Descent","created":"2022-07-08","cited_by_count":39,"area":null},{"id":"2302.01107","title":"A Survey on Efficient Training of Transformers","created":"2023-02-02","cited_by_count":39,"area":null},{"id":"2105.14573","title":"Embedding Principle of Loss Landscape of Deep Neural Networks","created":"2021-05-30","cited_by_count":38,"area":null},{"id":"2205.06226","title":"The Mechanism of Prediction Head in Non-contrastive Self-supervised Learning","created":"2022-05-12","cited_by_count":38,"area":null},{"id":"2211.16327","title":"On the Power of Foundation Models","created":"2022-11-29","cited_by_count":37,"area":null},{"id":"2301.12832","title":"Deep networks for system identification: a Survey","created":"2023-01-30","cited_by_count":37,"area":null},{"id":"2306.13575","title":"Scaling MLPs: A Tale of Inductive Bias","created":"2023-06-23","cited_by_count":37,"area":null},{"id":"2307.05014","title":"Test-Time Training on Video Streams","created":"2023-07-11","cited_by_count":37,"area":null},{"id":"2310.12680","title":"On the Optimization and Generalization of Multi-head Attention","created":"2023-10-19","cited_by_count":37,"area":null},{"id":"2106.07830","title":"On the Convergence and Calibration of Deep Learning with Differential Privacy","created":"2021-06-15","cited_by_count":36,"area":null},{"id":"2109.13236","title":"FedIPR: Ownership Verification for Federated Deep Neural Network Models","created":"2021-09-27","cited_by_count":36,"area":"safety"},{"id":"2111.12292","title":"Improved Fine-Tuning by Better Leveraging Pre-Training Data","created":"2021-11-24","cited_by_count":36,"area":null},{"id":"2205.10217","title":"Memorization and Optimization in Deep Neural Networks with Minimum Over-parameterization","created":"2022-05-20","cited_by_count":36,"area":null},{"id":"2202.07626","title":"Random Feature Amplification: Feature Learning and Generalization in Neural Networks","created":"2022-02-15","cited_by_count":35,"area":null},{"id":"2305.00660","title":"An Iterative Algorithm for Rescaled Hyperbolic Functions Regression","created":"2023-05-01","cited_by_count":35,"area":null},{"id":"2306.04073","title":"Patch-level Routing in Mixture-of-Experts is Provably Sample-efficient for Convolutional Neural Networks","created":"2023-06-07","cited_by_count":35,"area":"moe"}],"next_cursor":"100"}