{"total":190,"rows":[{"id":"2106.09685","title":"LoRA: Low-Rank Adaptation of Large Language Models","created":"2021-06-17","cited_by_count":14380,"area":null},{"id":"2303.10512","title":"AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning","created":"2023-03-18","cited_by_count":357,"area":null},{"id":"2203.03466","title":"Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer","created":"2022-03-07","cited_by_count":288,"area":null},{"id":"2403.09611","title":"MM1: Methods, Analysis & Insights from Multimodal LLM Pre-training","created":"2024-03-14","cited_by_count":232,"area":"mllm"},{"id":"2205.01445","title":"High-dimensional Asymptotics of Feature Learning: How One Gradient Step Improves the Representation","created":"2022-05-03","cited_by_count":150,"area":null},{"id":"2407.05000","title":"LoRA-GA: Low-Rank Adaptation with Gradient Approximation","created":"2024-07-06","cited_by_count":141,"area":null},{"id":"2311.12786","title":"Mechanistically analyzing the effects of fine-tuning on procedurally defined tasks","created":"2023-11-21","cited_by_count":115,"area":null},{"id":"2403.08540","title":"Language models scale reliably with over-training and on downstream tasks","created":"2024-03-13","cited_by_count":113,"area":null},{"id":"2310.17813","title":"A Spectral Condition for Feature Learning","created":"2023-10-26","cited_by_count":109,"area":null},{"id":"2111.00034","title":"Neural Networks as Kernel Learners: The Silent Alignment Effect","created":"2021-10-29","cited_by_count":106,"area":null},{"id":"2205.09653","title":"Self-Consistent Dynamical Field Theory of Kernel Evolution in Wide Neural Networks","created":"2022-05-19","cited_by_count":105,"area":null},{"id":"2105.06868","title":"Priors in Bayesian Deep Learning: A Review","created":"2021-05-14","cited_by_count":101,"area":null},{"id":"2106.15933","title":"Saddle-to-Saddle Dynamics in Deep Linear Networks: Small Initialization Training, Symmetry, and Sparsity","created":"2021-06-30","cited_by_count":85,"area":null},{"id":"2301.02679","title":"Grokking modular arithmetic","created":"2023-01-06","cited_by_count":83,"area":null},{"id":"2505.01618","title":"Don't be lazy: CompleteP enables compute-efficient deep transformers","created":"2025-05-02","cited_by_count":78,"area":null},{"id":"2205.13703","title":"Why So Pessimistic? Estimating Uncertainties for Offline RL through Ensembles, and Why Their Independence Matters","created":"2022-05-27","cited_by_count":76,"area":"offlinerl"},{"id":"2202.05928","title":"Benign Overfitting without Linearity: Neural Network Classifiers Trained by Gradient Descent for Noisy Linear Data","created":"2022-02-11","cited_by_count":74,"area":null},{"id":"2507.07955","title":"Dynamic Chunking for End-to-End Hierarchical Sequence Modeling","created":"2025-07-10","cited_by_count":72,"area":null},{"id":"2307.02129","title":"How Deep Neural Networks Learn Compositional Data: The Random Hierarchy Model","created":"2023-07-05","cited_by_count":71,"area":null},{"id":"2105.03703","title":"Tensor Programs IIb: Architectural Universality of Neural Tangent Kernel Training Dynamics","created":"2021-05-08","cited_by_count":57,"area":null},{"id":"2303.00055","title":"Learning time-scales in two-layers neural networks","created":"2023-02-28","cited_by_count":57,"area":null},{"id":"2503.02113","title":"Deep Learning is Not So Mysterious or Different","created":"2025-03-03","cited_by_count":56,"area":null},{"id":"2310.07891","title":"A Theory of Non-Linear Feature Learning with One Gradient Step in Two-Layer Neural Networks","created":"2023-10-11","cited_by_count":49,"area":null},{"id":"2409.14623","title":"From Lazy to Rich: Exact Learning Dynamics in Deep Linear Networks","created":"2024-09-22","cited_by_count":49,"area":null},{"id":"2306.17759","title":"The Shaped Transformer: Attention Models in the Infinite Depth-and-Width Limit","created":"2023-06-30","cited_by_count":47,"area":null},{"id":"2304.03408","title":"Dynamics of Finite Width Kernel and Prediction Fluctuations in Mean Field Neural Networks","created":"2023-04-06","cited_by_count":43,"area":null},{"id":"2505.13738","title":"Power Lines: Scaling Laws for Weight Decay and Batch Size in LLM Pre-training","created":"2025-05-19","cited_by_count":43,"area":null},{"id":"2406.06158","title":"Get rich quick: exact solutions reveal how unbalanced initializations promote rapid feature learning","created":"2024-06-10","cited_by_count":42,"area":null},{"id":"2206.02768","title":"The Neural Covariance SDE: Shaped Infinite Depth-and-Width Networks at Initialization","created":"2022-06-06","cited_by_count":41,"area":null},{"id":"2405.15712","title":"Infinite Limits of Multi-head Transformer Dynamics","created":"2024-05-24","cited_by_count":39,"area":null},{"id":"2406.00048","title":"Towards a theory of how the structure of language is acquired by deep neural networks","created":"2024-05-28","cited_by_count":39,"area":null},{"id":"2202.07626","title":"Random Feature Amplification: Feature Learning and Generalization in Neural Networks","created":"2022-02-15","cited_by_count":35,"area":null},{"id":"2502.15938","title":"Straight to Zero: Why Linearly Decaying the Learning Rate to Zero Works Best for LLMs","created":"2025-02-21","cited_by_count":35,"area":null},{"id":"2503.04715","title":"Predictable Scale: Part I, Step Law -- Optimal Hyperparameter Scaling Law in Large Language Model Pretraining","created":"2025-03-06","cited_by_count":34,"area":null},{"id":"2106.04013","title":"The Future is Log-Gaussian: ResNets and Their Infinite-Depth-and-Width Limit at Initialization","created":"2021-06-07","cited_by_count":33,"area":null},{"id":"2106.04110","title":"A self consistent theory of Gaussian Processes captures feature learning effects in finite CNNs","created":"2021-06-08","cited_by_count":33,"area":null},{"id":"2306.04815","title":"Catapults in SGD: spikes in the training loss and their impact on generalization through feature learning","created":"2023-06-07","cited_by_count":31,"area":null},{"id":"2211.06506","title":"Spectral Evolution and Invariance in Linear-width Neural Networks","created":"2022-11-11","cited_by_count":30,"area":null},{"id":"2406.06248","title":"Compute Better Spent: Replacing Dense Layers with Structured Matrices","created":"2024-06-10","cited_by_count":28,"area":null},{"id":"2408.13359","title":"Power Scheduler: A Batch Size and Token Number Agnostic Learning Rate Scheduler","created":"2024-08-23","cited_by_count":28,"area":null},{"id":"2105.10585","title":"Properties of the After Kernel","created":"2021-05-21","cited_by_count":27,"area":null},{"id":"2402.10127","title":"Nonlinear spiked covariance matrices and signal propagation in deep neural networks","created":"2024-02-15","cited_by_count":26,"area":null},{"id":"2105.14301","title":"A Theory of Neural Tangent Kernel Alignment and Its Influence on Training","created":"2021-05-29","cited_by_count":25,"area":null},{"id":"2212.12147","title":"The Onset of Variance-Limited Behavior for Networks in the Lazy and Rich Regimes","created":"2022-12-23","cited_by_count":25,"area":null},{"id":"2405.21064","title":"Recurrent neural networks: vanishing and exploding gradients are not the end of the story","created":"2024-05-31","cited_by_count":25,"area":null},{"id":"2502.12089","title":"How Compositional Generalization and Creativity Improve as Diffusion Models are Trained","created":"2025-02-17","cited_by_count":25,"area":"diffusion"},{"id":"2505.16959","title":"Bigger Isn't Always Memorizing: Early Stopping Overparameterized Diffusion Models","created":"2025-05-22","cited_by_count":25,"area":"diffusion"},{"id":"2109.13247","title":"The edge of chaos: quantum field theory and deep neural networks","created":"2021-09-27","cited_by_count":24,"area":null},{"id":"2205.14846","title":"Precise Learning Curves and Higher-Order Scaling Limits for Dot Product Kernel Regression","created":"2022-05-30","cited_by_count":22,"area":null},{"id":"2507.02119","title":"Scaling Collapse Reveals Universal Dynamics in Compute-Optimally Trained Neural Networks","created":"2025-07-02","cited_by_count":22,"area":null},{"id":"2106.05566","title":"A Neural Tangent Kernel Perspective of GANs","created":"2021-06-10","cited_by_count":21,"area":null},{"id":"2106.06529","title":"The Limitations of Large Width in Neural Networks: A Deep Gaussian Process Perspective","created":"2021-06-11","cited_by_count":21,"area":null},{"id":"2203.14661","title":"Random matrix analysis of deep neural network weight matrices","created":"2022-03-28","cited_by_count":21,"area":null},{"id":"2203.16711","title":"Analytic theory for the dynamics of wide quantum neural networks","created":"2022-03-30","cited_by_count":21,"area":null},{"id":"2512.05620","title":"Hyperparameter Transfer Enables Consistent Gains of Matrix-Preconditioned Optimizers Across Scales","created":"2025-12-05","cited_by_count":21,"area":null},{"id":"2308.13431","title":"Six Lectures on Linearized Neural Networks","created":"2023-08-25","cited_by_count":20,"area":null},{"id":"2401.07187","title":"A Survey on Statistical Theory of Deep Learning: Approximation, Training Dynamics, and Generative Models","created":"2024-01-14","cited_by_count":20,"area":null},{"id":"2302.05779","title":"How to prepare your task head for finetuning","created":"2023-02-11","cited_by_count":19,"area":null},{"id":"2403.03103","title":"Emergent Equivariance in Deep Ensembles","created":"2024-03-05","cited_by_count":18,"area":null},{"id":"2405.17580","title":"Mixed Dynamics In Linear Networks: Unifying the Lazy and Active Regimes","created":"2024-05-27","cited_by_count":18,"area":null},{"id":"2111.06063","title":"On the Equivalence between Neural Network and Support Vector Machine","created":"2021-11-11","cited_by_count":17,"area":null},{"id":"2112.05611","title":"Eigenspace Restructuring: a Principle of Space and Frequency in Neural Networks","created":"2021-12-10","cited_by_count":17,"area":null},{"id":"2205.11787","title":"Quadratic models for understanding catapult dynamics of neural networks","created":"2022-05-24","cited_by_count":17,"area":null},{"id":"2205.15809","title":"Feature Learning in $L_{2}$-regularized DNNs: Attraction/Repulsion and Sparsity","created":"2022-05-31","cited_by_count":17,"area":null},{"id":"2210.06545","title":"GULP: a prediction-based metric between representations","created":"2022-10-12","cited_by_count":17,"area":null},{"id":"2303.11257","title":"Unit Scaling: Out-of-the-Box Low-Precision Training","created":"2023-03-20","cited_by_count":17,"area":null},{"id":"2402.16828","title":"Training Neural Networks from Scratch with Parallel Low-Rank Adapters","created":"2024-02-26","cited_by_count":17,"area":null},{"id":"2505.07067","title":"Learning curves theory for hierarchically compositional data with power-law distributed features","created":"2025-05-11","cited_by_count":17,"area":null},{"id":"2505.24333","title":"Two failure modes of deep transformers and how to avoid them: a unified theory of signal propagation at initialisation","created":"2025-05-30","cited_by_count":17,"area":null},{"id":"2110.08084","title":"Gradient Descent on Infinitely Wide Neural Networks: Global Convergence and Generalization","created":"2021-10-15","cited_by_count":16,"area":null},{"id":"2405.15743","title":"Sparse maximal update parameterization: A holistic approach to sparse training dynamics","created":"2024-05-24","cited_by_count":16,"area":null},{"id":"2108.13097","title":"A theory of representation learning gives a deep generalisation of kernel methods","created":"2021-08-30","cited_by_count":15,"area":null},{"id":"2405.03251","title":"Exploring the Frontiers of Softmax: Provable Optimization, Applications in Diffusion Model, and Beyond","created":"2024-05-06","cited_by_count":15,"area":"diffusion"},{"id":"2210.12030","title":"Evolution of Neural Tangent Kernels under Benign and Adversarial Training","created":"2022-10-21","cited_by_count":14,"area":null},{"id":"2310.12408","title":"Provable Guarantees for Neural Networks via Gradient Feature Learning","created":"2023-10-19","cited_by_count":14,"area":null},{"id":"2506.06489","title":"Alternating Gradient Flows: A Theory of Feature Learning in Two-layer Neural Networks","created":"2025-06-06","cited_by_count":14,"area":null},{"id":"2506.20629","title":"PLoP: Precise LoRA Placement for Efficient Finetuning of Large Models","created":"2025-06-25","cited_by_count":14,"area":null},{"id":"2106.08453","title":"How to Train Your Wide Neural Network Without Backprop: An Input-Weight Alignment Perspective","created":"2021-06-15","cited_by_count":13,"area":null},{"id":"2301.13310","title":"Alternating Updates for Efficient Transformers","created":"2023-01-30","cited_by_count":13,"area":null},{"id":"2406.11828","title":"Learning sum of diverse features: computational hardness and efficient gradient-based training for ridge combinations","created":"2024-06-17","cited_by_count":13,"area":null},{"id":"2410.02117","title":"Searching for Efficient Linear Layers over a Continuous Space of Structured Matrices","created":"2024-10-03","cited_by_count":13,"area":"moe"},{"id":"2306.13926","title":"Quantifying the Optimization and Generalization Advantages of Graph Neural Networks Over Multilayer Perceptrons","created":"2023-06-24","cited_by_count":12,"area":null},{"id":"2502.05300","title":"Parameter Symmetry Potentially Unifies Deep Learning Theory","created":"2025-02-07","cited_by_count":12,"area":null},{"id":"2102.00940","title":"Meta-learning with negative learning rates","created":"2021-02-01","cited_by_count":11,"area":null},{"id":"2311.18718","title":"The Feature Speed Formula: a flexible approach to scale hyper-parameters of deep neural networks","created":"2023-11-30","cited_by_count":11,"area":null},{"id":"2502.02870","title":"Uncertainty Quantification with the Empirical Neural Tangent Kernel","created":"2025-02-05","cited_by_count":11,"area":null},{"id":"2202.11104","title":"A duality connecting neural network and cosmological dynamics","created":"2022-02-22","cited_by_count":10,"area":null},{"id":"2206.09313","title":"Laziness, Barren Plateau, and Noise in Machine Learning","created":"2022-06-19","cited_by_count":10,"area":null},{"id":"2402.05271","title":"Feature learning as alignment: a structural property of gradient descent in non-linear neural networks","created":"2024-02-07","cited_by_count":10,"area":null},{"id":"2405.19279","title":"Understanding and Minimising Outlier Features in Neural Network Training","created":"2024-05-29","cited_by_count":10,"area":null},{"id":"2410.03006","title":"Formation of Representations in Neural Networks","created":"2024-10-03","cited_by_count":10,"area":null},{"id":"2412.11521","title":"On the Ability of Deep Networks to Learn Symmetries from Data: A Neural Kernel Theory","created":"2024-12-16","cited_by_count":10,"area":null},{"id":"2502.03210","title":"From Kernels to Features: A Multi-Scale Adaptive Theory of Feature Learning","created":"2025-02-05","cited_by_count":10,"area":null},{"id":"2405.17538","title":"Bayesian RG Flow in Neural Network Field Theories","created":"2024-05-27","cited_by_count":9,"area":null},{"id":"2410.05838","title":"Time Transfer: On Optimal Learning Rate and Batch Size In The Infinite Data Limit","created":"2024-10-08","cited_by_count":9,"area":null},{"id":"2411.08212","title":"PERFT: Parameter-Efficient Routed Fine-Tuning for Mixture-of-Expert Model","created":"2024-11-12","cited_by_count":9,"area":"moe"},{"id":"2505.12455","title":"AltLoRA: Towards Better Gradient Approximation in Low-Rank Adaptation with Alternating Projections","created":"2025-05-18","cited_by_count":9,"area":null},{"id":"2509.25087","title":"Scaling with Collapse: Efficient and Predictable Training of LLM Families","created":"2025-09-29","cited_by_count":9,"area":null},{"id":"2602.06204","title":"Learning Rate Scaling across LoRA Ranks and Transfer to Full Finetuning","created":"2026-02-05","cited_by_count":9,"area":null},{"id":"2501.12391","title":"Physics of Skill Learning","created":"2025-01-21","cited_by_count":8,"area":null}],"next_cursor":"100"}