{"total":195,"rows":[{"id":"2402.12354","title":"LoRA+: Efficient Low Rank Adaptation of Large Models","created":"2024-02-19","cited_by_count":377,"area":null},{"id":"2102.06701","title":"Explaining Neural Scaling Laws","created":"2021-02-12","cited_by_count":363,"area":null},{"id":"2203.03466","title":"Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer","created":"2022-03-07","cited_by_count":288,"area":null},{"id":"2303.01486","title":"Understanding plasticity in neural networks","created":"2023-03-02","cited_by_count":174,"area":null},{"id":"2210.01117","title":"Omnigrok: Grokking Beyond Algorithmic Data","created":"2022-10-03","cited_by_count":146,"area":null},{"id":"2206.03126","title":"Signal Propagation in Transformers: Theoretical Perspectives and the Role of Rank Collapse","created":"2022-06-07","cited_by_count":141,"area":null},{"id":"2304.03208","title":"Cerebras-GPT: Open Compute-Optimal Language Models Trained on the Cerebras Wafer-Scale Cluster","created":"2023-04-06","cited_by_count":109,"area":null},{"id":"2205.09653","title":"Self-Consistent Dynamical Field Theory of Kernel Evolution in Wide Neural Networks","created":"2022-05-19","cited_by_count":105,"area":null},{"id":"2110.01654","title":"Improved architectures and training algorithms for deep operator networks","created":"2021-10-04","cited_by_count":86,"area":null},{"id":"2402.18762","title":"Disentangling the Causes of Plasticity Loss in Neural Networks","created":"2024-02-29","cited_by_count":86,"area":"deeprl"},{"id":"2505.01618","title":"Don't be lazy: CompleteP enables compute-efficient deep transformers","created":"2025-05-02","cited_by_count":78,"area":null},{"id":"2206.08720","title":"Fast Finite Width Neural Tangent Kernel","created":"2022-06-17","cited_by_count":62,"area":null},{"id":"2406.08447","title":"The Impact of Initialization on LoRA Finetuning Dynamics","created":"2024-06-12","cited_by_count":54,"area":null},{"id":"2306.17759","title":"The Shaped Transformer: Attention Models in the Infinite Depth-and-Width Limit","created":"2023-06-30","cited_by_count":47,"area":null},{"id":"2202.00089","title":"Understanding AdamW through Proximal Methods and Scale-Freeness","created":"2022-01-31","cited_by_count":46,"area":null},{"id":"2302.10322","title":"Deep Transformers without Shortcuts: Modifying Self-attention for Faithful Signal Propagation","created":"2023-02-20","cited_by_count":45,"area":null},{"id":"2403.02241","title":"Neural Redshift: Random Networks are not Random Functions","created":"2024-03-04","cited_by_count":43,"area":null},{"id":"2206.02768","title":"The Neural Covariance SDE: Shaped Infinite Depth-and-Width Networks at Initialization","created":"2022-06-06","cited_by_count":41,"area":null},{"id":"2306.13575","title":"Scaling MLPs: A Tale of Inductive Bias","created":"2023-06-23","cited_by_count":37,"area":null},{"id":"2311.01906","title":"Simplifying Transformer Blocks","created":"2023-11-03","cited_by_count":37,"area":null},{"id":"2202.00553","title":"Neural Tangent Kernel Beyond the Infinite-Width Limit: Effects of Depth and Initialization","created":"2022-02-01","cited_by_count":30,"area":null},{"id":"2209.11208","title":"A Closer Look at Learned Optimization: Stability, Robustness, and Inductive Biases","created":"2022-09-22","cited_by_count":30,"area":null},{"id":"2106.03970","title":"Batch Normalization Orthogonalizes Representations in Deep Random Networks","created":"2021-06-07","cited_by_count":29,"area":null},{"id":"2106.06615","title":"Precise characterization of the prior predictive distribution of deep ReLU networks","created":"2021-06-11","cited_by_count":28,"area":null},{"id":"2203.08120","title":"Deep Learning without Shortcuts: Shaping the Kernel with Tailored Rectifiers","created":"2022-03-15","cited_by_count":28,"area":null},{"id":"2304.06670","title":"Deep neural networks have an inbuilt Occam's razor","created":"2023-04-13","cited_by_count":28,"area":null},{"id":"2309.16314","title":"A Primer on Bayesian Neural Networks: Review and Debates","created":"2023-09-28","cited_by_count":27,"area":null},{"id":"2106.05956","title":"Beyond BatchNorm: Towards a Unified Understanding of Normalization in Deep Learning","created":"2021-06-10","cited_by_count":25,"area":null},{"id":"2212.13881","title":"Mechanism of feature learning in deep fully connected networks and kernel machines that recursively learn features","created":"2022-12-28","cited_by_count":25,"area":null},{"id":"2303.17671","title":"Neural signature kernels as infinite-width-depth-limits of controlled ResNets","created":"2023-03-30","cited_by_count":25,"area":null},{"id":"2109.13247","title":"The edge of chaos: quantum field theory and deep neural networks","created":"2021-09-27","cited_by_count":24,"area":null},{"id":"2506.15544","title":"Stable Gradients for Stable Learning at Scale in Deep Reinforcement Learning","created":"2025-06-18","cited_by_count":23,"area":"deeprl"},{"id":"2210.00688","title":"On the infinite-depth limit of finite-width neural networks","created":"2022-10-03","cited_by_count":22,"area":null},{"id":"2211.07844","title":"Characterizing the Spectrum of the NTK via a Power Series Expansion","created":"2022-11-15","cited_by_count":22,"area":null},{"id":"2109.09304","title":"Deformed semicircle law and concentration of nonlinear random matrices for ultra-wide neural networks","created":"2021-09-20","cited_by_count":21,"area":null},{"id":"2302.00453","title":"Width and Depth Limits Commute in Residual Networks","created":"2023-02-01","cited_by_count":21,"area":null},{"id":"2304.02034","title":"Effective Theory of Transformers at Initialization","created":"2023-04-04","cited_by_count":21,"area":null},{"id":"2512.05620","title":"Hyperparameter Transfer Enables Consistent Gains of Matrix-Preconditioned Optimizers Across Scales","created":"2025-12-05","cited_by_count":21,"area":null},{"id":"2307.06092","title":"Quantitative CLTs in Deep Neural Networks","created":"2023-07-12","cited_by_count":20,"area":null},{"id":"2309.17310","title":"Leave-one-out Distinguishability in Machine Learning","created":"2023-09-29","cited_by_count":20,"area":null},{"id":"2107.09133","title":"The Limiting Dynamics of SGD: Modified Loss, Phase Space Oscillations, and Anomalous Diffusion","created":"2021-07-19","cited_by_count":19,"area":null},{"id":"2210.04909","title":"Meta-Principled Family of Hyperparameter Scaling Strategies","created":"2022-10-10","cited_by_count":19,"area":null},{"id":"2305.06986","title":"Provable Guarantees for Nonlinear Feature Learning in Three-Layer Neural Networks","created":"2023-05-11","cited_by_count":19,"area":null},{"id":"2110.11749","title":"Feature Learning and Signal Propagation in Deep Neural Networks","created":"2021-10-22","cited_by_count":18,"area":null},{"id":"2112.05611","title":"Eigenspace Restructuring: a Principle of Space and Frequency in Neural Networks","created":"2021-12-10","cited_by_count":17,"area":null},{"id":"2505.23194","title":"Beyond Zero Initialization: Investigating the Impact of Non-Zero Initialization on LoRA Fine-Tuning Dynamics","created":"2025-05-29","cited_by_count":16,"area":null},{"id":"2107.12530","title":"Convergence of Deep ReLU Networks","created":"2021-07-27","cited_by_count":15,"area":null},{"id":"2312.12226","title":"On the Parameterization of Second-Order Optimization Effective Towards the Infinite Width","created":"2023-12-19","cited_by_count":15,"area":null},{"id":"2103.03113","title":"Towards Deepening Graph Neural Networks: A GNTK-based Optimization Perspective","created":"2021-03-03","cited_by_count":14,"area":null},{"id":"2107.06898","title":"Towards quantifying information flows: relative entropy in deep neural networks and the renormalization group","created":"2021-07-14","cited_by_count":14,"area":null},{"id":"2310.10767","title":"Wide Neural Networks as Gaussian Processes: Lessons from Deep Equilibrium Models","created":"2023-10-16","cited_by_count":14,"area":null},{"id":"2105.07741","title":"Activation function design for deep networks: linearity and effective initialisation","created":"2021-05-17","cited_by_count":13,"area":null},{"id":"2305.02334","title":"Structures of Neural Network Effective Theories","created":"2023-05-03","cited_by_count":13,"area":null},{"id":"2305.18399","title":"On the impact of activation and normalization in obtaining isometric embeddings at initialization","created":"2023-05-28","cited_by_count":13,"area":null},{"id":"2106.03186","title":"Reverse Engineering the Neural Tangent Kernel","created":"2021-06-06","cited_by_count":12,"area":null},{"id":"2202.04925","title":"Decomposing neural networks as mappings of correlation functions","created":"2022-02-10","cited_by_count":12,"area":null},{"id":"2205.08099","title":"Dimensionality Reduced Training by Pruning and Freezing Parts of a Deep Neural Network, a Survey","created":"2022-05-17","cited_by_count":12,"area":null},{"id":"2312.12474","title":"Principled Weight Initialisation for Input-Convex Neural Networks","created":"2023-12-19","cited_by_count":12,"area":null},{"id":"2506.01115","title":"Is Random Attention Sufficient for Sequence Modeling? Disentangling Trainable Components in the Transformer","created":"2025-06-01","cited_by_count":12,"area":"lr"},{"id":"2210.08122","title":"Old can be Gold: Better Gradient Flow can Make Vanilla-GCNs Great Again","created":"2022-10-14","cited_by_count":11,"area":null},{"id":"2310.06379","title":"Understanding the Expressivity and Trainability of Fourier Neural Operator: A Mean-Field Perspective","created":"2023-10-10","cited_by_count":11,"area":null},{"id":"2106.09798","title":"Wide stochastic networks: Gaussian limit and PAC-Bayesian training","created":"2021-06-17","cited_by_count":10,"area":null},{"id":"2405.19279","title":"Understanding and Minimising Outlier Features in Neural Network Training","created":"2024-05-29","cited_by_count":10,"area":null},{"id":"2502.03210","title":"From Kernels to Features: A Multi-Scale Adaptive Theory of Feature Learning","created":"2025-02-05","cited_by_count":10,"area":null},{"id":"2205.12379","title":"Gaussian Pre-Activations in Neural Networks: Myth or Reality?","created":"2022-05-24","cited_by_count":9,"area":null},{"id":"2405.17538","title":"Bayesian RG Flow in Neural Network Field Theories","created":"2024-05-27","cited_by_count":9,"area":null},{"id":"2410.02247","title":"Theoretical Insights into Fine-Tuning Attention Mechanism: Generalization and Optimization","created":"2024-10-03","cited_by_count":9,"area":null},{"id":"2505.12455","title":"AltLoRA: Towards Better Gradient Approximation in Low-Rank Adaptation with Alternating Projections","created":"2025-05-18","cited_by_count":9,"area":null},{"id":"2108.02316","title":"Deep Stable neural networks: large-width asymptotics and convergence rates","created":"2021-08-02","cited_by_count":8,"area":null},{"id":"2111.12143","title":"Critical Initialization of Wide and Deep Neural Networks through Partial Jacobians: General Theory and Applications","created":"2021-11-23","cited_by_count":8,"area":null},{"id":"2210.09809","title":"Analysis of Convolutions, Non-linearity and Depth in Graph Neural Networks using Neural Tangent Kernel","created":"2022-10-18","cited_by_count":8,"area":null},{"id":"2104.00008","title":"Why is AI hard and Physics simple?","created":"2021-03-31","cited_by_count":7,"area":null},{"id":"2106.03091","title":"Regularization in ResNet with Stochastic Depth","created":"2021-06-06","cited_by_count":7,"area":null},{"id":"2211.06077","title":"Overparameterized random feature regression with nearly orthogonal data","created":"2022-11-11","cited_by_count":7,"area":null},{"id":"2407.04117","title":"Predictive Coding Networks and Inference Learning: Tutorial and Survey","created":"2024-07-04","cited_by_count":7,"area":null},{"id":"2207.04612","title":"Synergy and Symmetry in Deep Learning: Interactions between the Data, Model, and Inference Algorithm","created":"2022-07-11","cited_by_count":6,"area":null},{"id":"2305.07715","title":"Field theory for optimal signal propagation in ResNets","created":"2023-05-12","cited_by_count":6,"area":null},{"id":"2305.08117","title":"MBQuant: A Novel Multi-Branch Topology Method for Arbitrary Bit-width Network Quantization","created":"2023-05-14","cited_by_count":6,"area":null},{"id":"2305.08813","title":"Better NTK Conditioning: A Free Lunch from (ReLU) Nonlinear Activation in Wide Neural Networks","created":"2023-05-15","cited_by_count":6,"area":null},{"id":"2403.19863","title":"DeNetDM: Debiasing by Network Depth Modulation","created":"2024-03-28","cited_by_count":6,"area":null},{"id":"2506.15025","title":"Optimal Embedding Learning Rate in LLMs: The Effect of Vocabulary Size","created":"2025-06-17","cited_by_count":6,"area":null},{"id":"2103.13466","title":"Asymptotic Freeness of Layerwise Jacobians Caused by Invariance of Multilayer Perceptron: The Haar Orthogonal Case","created":"2021-03-24","cited_by_count":5,"area":null},{"id":"2110.02885","title":"Bayesian neural network unit priors and generalized Weibull-tail property","created":"2021-10-06","cited_by_count":5,"area":null},{"id":"2202.00339","title":"Quantifying Relevance in Learning and Inference","created":"2022-02-01","cited_by_count":5,"area":null},{"id":"2208.06028","title":"Gaussian Process Surrogate Models for Neural Networks","created":"2022-08-11","cited_by_count":5,"area":null},{"id":"2210.01506","title":"How deep convolutional neural networks lose spatial information with training","created":"2022-10-04","cited_by_count":5,"area":null},{"id":"2301.13710","title":"On the Initialisation of Wide Low-Rank Feedforward Neural Networks","created":"2023-01-31","cited_by_count":5,"area":null},{"id":"2306.00809","title":"Initial Guessing Bias: How Untrained Networks Favor Some Classes","created":"2023-06-01","cited_by_count":5,"area":null},{"id":"2402.16184","title":"Deep Neural Network Initialization with Sparsity Inducing Activations","created":"2024-02-25","cited_by_count":5,"area":null},{"id":"2403.04747","title":"GNN-VPA: A Variance-Preserving Aggregation Strategy for Graph Neural Networks","created":"2024-03-07","cited_by_count":5,"area":null},{"id":"2406.18630","title":"Improving Hyperparameter Optimization with Checkpointed Model Weights","created":"2024-06-26","cited_by_count":5,"area":null},{"id":"2408.00082","title":"TASI Lectures on Physics for Machine Learning","created":"2024-07-31","cited_by_count":5,"area":null},{"id":"2506.00846","title":"Infinite-Width Limit of a Single Attention Layer: Analysis via Tensor Programs","created":"2025-06-01","cited_by_count":5,"area":null},{"id":"2602.18308","title":"JPmHC Dynamical Isometry via Orthogonal Hyper-Connections","created":"2026-02-20","cited_by_count":5,"area":null},{"id":"2605.14258","title":"Dynamics of the Transformer Residual Stream: Coupling Spectral Geometry to Network Topology","created":"2026-05-14","cited_by_count":5,"area":null},{"id":"2110.11150","title":"Lottery Tickets with Nonzero Biases","created":"2021-10-21","cited_by_count":4,"area":null},{"id":"2201.04543","title":"Eigenvalue Distribution of Large Random Matrices Arising in Deep Neural Networks: Orthogonal Case","created":"2022-01-12","cited_by_count":4,"area":null},{"id":"2307.02284","title":"Universal Scaling Laws of Absorbing Phase Transitions in Artificial Deep Neural Networks","created":"2023-07-05","cited_by_count":4,"area":null},{"id":"2403.20284","title":"LayerNorm: A key component in parameter-efficient fine-tuning","created":"2024-03-29","cited_by_count":4,"area":null},{"id":"2503.08231","title":"How good is PAC-Bayes at explaining generalisation?","created":"2025-03-11","cited_by_count":4,"area":null}],"next_cursor":"100"}