{"total":83,"rows":[{"id":"2203.03466","title":"Tensor Programs V: Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer","created":"2022-03-07","cited_by_count":288,"area":null},{"id":"2106.01548","title":"When Vision Transformers Outperform ResNets without Pre-training or Strong Data Augmentations","created":"2021-06-03","cited_by_count":260,"area":null},{"id":"2303.01486","title":"Understanding plasticity in neural networks","created":"2023-03-02","cited_by_count":174,"area":null},{"id":"2102.11535","title":"Neural Architecture Search on ImageNet in Four GPU Hours: A Theoretically Inspired Perspective","created":"2021-02-23","cited_by_count":169,"area":null},{"id":"2210.01117","title":"Omnigrok: Grokking Beyond Algorithmic Data","created":"2022-10-03","cited_by_count":146,"area":null},{"id":"2304.03208","title":"Cerebras-GPT: Open Compute-Optimal Language Models Trained on the Cerebras Wafer-Scale Cluster","created":"2023-04-06","cited_by_count":109,"area":null},{"id":"2205.09653","title":"Self-Consistent Dynamical Field Theory of Kernel Evolution in Wide Neural Networks","created":"2022-05-19","cited_by_count":105,"area":null},{"id":"2101.08692","title":"Characterizing signal propagation to close the performance gap in unnormalized ResNets","created":"2021-01-21","cited_by_count":88,"area":null},{"id":"2111.05803","title":"Gradients are Not All You Need","created":"2021-11-10","cited_by_count":86,"area":null},{"id":"2104.14129","title":"ActNN: Reducing Training Memory Footprint via 2-Bit Activation Compressed Training","created":"2021-04-29","cited_by_count":61,"area":null},{"id":"2105.01029","title":"Initialization and Regularization of Factorized Neural Layers","created":"2021-05-03","cited_by_count":58,"area":"distill"},{"id":"2105.03703","title":"Tensor Programs IIb: Architectural Universality of Neural Tangent Kernel Training Dynamics","created":"2021-05-08","cited_by_count":57,"area":null},{"id":"2306.17759","title":"The Shaped Transformer: Attention Models in the Infinite Depth-and-Width Limit","created":"2023-06-30","cited_by_count":47,"area":null},{"id":"2206.02768","title":"The Neural Covariance SDE: Shaped Infinite Depth-and-Width Networks at Initialization","created":"2022-06-06","cited_by_count":41,"area":null},{"id":"2106.04013","title":"The Future is Log-Gaussian: ResNets and Their Infinite-Depth-and-Width Limit at Initialization","created":"2021-06-07","cited_by_count":33,"area":null},{"id":"2202.00553","title":"Neural Tangent Kernel Beyond the Infinite-Width Limit: Effects of Depth and Initialization","created":"2022-02-01","cited_by_count":30,"area":null},{"id":"2209.11208","title":"A Closer Look at Learned Optimization: Stability, Robustness, and Inductive Biases","created":"2022-09-22","cited_by_count":30,"area":null},{"id":"2203.08120","title":"Deep Learning without Shortcuts: Shaping the Kernel with Tailored Rectifiers","created":"2022-03-15","cited_by_count":28,"area":null},{"id":"2205.14612","title":"Do Residual Neural Networks discretize Neural Ordinary Differential Equations?","created":"2022-05-29","cited_by_count":27,"area":null},{"id":"2212.13881","title":"Mechanism of feature learning in deep fully connected networks and kernel machines that recursively learn features","created":"2022-12-28","cited_by_count":25,"area":null},{"id":"2210.00688","title":"On the infinite-depth limit of finite-width neural networks","created":"2022-10-03","cited_by_count":22,"area":null},{"id":"2302.00453","title":"Width and Depth Limits Commute in Residual Networks","created":"2023-02-01","cited_by_count":21,"area":null},{"id":"2110.11749","title":"Feature Learning and Signal Propagation in Deep Neural Networks","created":"2021-10-22","cited_by_count":18,"area":null},{"id":"2206.06929","title":"Scaling ResNets in the Large-depth Regime","created":"2022-06-14","cited_by_count":18,"area":null},{"id":"2505.24333","title":"Two failure modes of deep transformers and how to avoid them: a unified theory of signal propagation at initialisation","created":"2025-05-30","cited_by_count":17,"area":null},{"id":"2103.03113","title":"Towards Deepening Graph Neural Networks: A GNTK-based Optimization Perspective","created":"2021-03-03","cited_by_count":14,"area":null},{"id":"2302.06079","title":"Byzantine-Robust Learning on Heterogeneous Data via Gradient Splitting","created":"2023-02-13","cited_by_count":14,"area":null},{"id":"2204.14126","title":"Wide and Deep Neural Networks Achieve Optimality for Classification","created":"2022-04-29","cited_by_count":12,"area":null},{"id":"2312.12474","title":"Principled Weight Initialisation for Input-Convex Neural Networks","created":"2023-12-19","cited_by_count":12,"area":null},{"id":"2506.01115","title":"Is Random Attention Sufficient for Sequence Modeling? Disentangling Trainable Components in the Transformer","created":"2025-06-01","cited_by_count":12,"area":"lr"},{"id":"2310.06379","title":"Understanding the Expressivity and Trainability of Fourier Neural Operator: A Mean-Field Perspective","created":"2023-10-10","cited_by_count":11,"area":null},{"id":"2111.12143","title":"Critical Initialization of Wide and Deep Neural Networks through Partial Jacobians: General Theory and Applications","created":"2021-11-23","cited_by_count":8,"area":null},{"id":"2210.09809","title":"Analysis of Convolutions, Non-linearity and Depth in Graph Neural Networks using Neural Tangent Kernel","created":"2022-10-18","cited_by_count":8,"area":null},{"id":"2601.21942","title":"Clustering in Deep Stochastic Transformers","created":"2026-01-29","cited_by_count":8,"area":null},{"id":"2106.03091","title":"Regularization in ResNet with Stochastic Depth","created":"2021-06-06","cited_by_count":7,"area":null},{"id":"2210.05371","title":"On skip connections and normalisation layers in deep optimisation","created":"2022-10-10","cited_by_count":7,"area":null},{"id":"2305.07715","title":"Field theory for optimal signal propagation in ResNets","created":"2023-05-12","cited_by_count":6,"area":null},{"id":"2210.01245","title":"Random orthogonal additive filters: a solution to the vanishing/exploding gradient of deep neural networks","created":"2022-10-03","cited_by_count":5,"area":null},{"id":"2301.13710","title":"On the Initialisation of Wide Low-Rank Feedforward Neural Networks","created":"2023-01-31","cited_by_count":5,"area":null},{"id":"2402.16184","title":"Deep Neural Network Initialization with Sparsity Inducing Activations","created":"2024-02-25","cited_by_count":5,"area":null},{"id":"2503.04626","title":"IDInit: A Universal and Stable Initialization Method for Neural Network Training","created":"2025-03-06","cited_by_count":5,"area":null},{"id":"2506.00846","title":"Infinite-Width Limit of a Single Attention Layer: Analysis via Tensor Programs","created":"2025-06-01","cited_by_count":5,"area":null},{"id":"2602.18308","title":"JPmHC Dynamical Isometry via Orthogonal Hyper-Connections","created":"2026-02-20","cited_by_count":5,"area":null},{"id":"2206.01261","title":"Entangled Residual Mappings","created":"2022-06-02","cited_by_count":4,"area":null},{"id":"2307.02284","title":"Universal Scaling Laws of Absorbing Phase Transitions in Artificial Deep Neural Networks","created":"2023-07-05","cited_by_count":4,"area":null},{"id":"2402.17440","title":"Principled Architecture-aware Scaling of Hyperparameters","created":"2024-02-27","cited_by_count":4,"area":null},{"id":"2111.00841","title":"Free Probability for predicting the performance of feed-forward fully connected neural networks","created":"2021-11-01","cited_by_count":3,"area":null},{"id":"2204.07261","title":"Convergence and Implicit Regularization Properties of Gradient Descent for Deep Residual Networks","created":"2022-04-14","cited_by_count":3,"area":null},{"id":"2210.05212","title":"On Scrambling Phenomena for Randomly Initialized Recurrent Networks","created":"2022-10-11","cited_by_count":3,"area":null},{"id":"2410.07451","title":"Collective variables of neural networks: empirical time evolution and scaling laws","created":"2024-10-09","cited_by_count":3,"area":null},{"id":"2209.08617","title":"PIM-QAT: Neural Network Quantization for Processing-In-Memory (PIM) Systems","created":"2022-09-18","cited_by_count":2,"area":null},{"id":"2212.01744","title":"Statistical Physics of Deep Neural Networks: Initialization toward Optimal Channels","created":"2022-12-04","cited_by_count":2,"area":null},{"id":"2307.14823","title":"Fading memory as inductive bias in residual recurrent networks","created":"2023-07-27","cited_by_count":2,"area":null},{"id":"2405.15903","title":"UnitNorm: Rethinking Normalization for Transformers in Time Series","created":"2024-05-24","cited_by_count":2,"area":null},{"id":"2410.15578","title":"Generalized Probabilistic Attention Mechanism in Transformers","created":"2024-10-21","cited_by_count":2,"area":null},{"id":"2602.05779","title":"How Controlling the Variance can Improve Training Stability of Sparsely Activated DNNs and CNNs","created":"2026-02-05","cited_by_count":2,"area":null},{"id":"2605.14685","title":"Spontaneous symmetry breaking and Goldstone modes for deep information propagation","created":"2026-05-14","cited_by_count":2,"area":null},{"id":"2102.04896","title":"Formalising the Use of the Activation Function in Neural Inference","created":"2021-02-02","cited_by_count":1,"area":null},{"id":"2310.01683","title":"Commutative Width and Depth Scaling in Deep Neural Networks","created":"2023-10-02","cited_by_count":1,"area":null},{"id":"2412.09853","title":"Understand the Effectiveness of Shortcuts through the Lens of DCA","created":"2024-12-13","cited_by_count":1,"area":null},{"id":"2509.22649","title":"Toward a Physics of Deep Learning and Brains","created":"2025-09-26","cited_by_count":1,"area":null},{"id":"2512.06427","title":"A new initialisation to Control Gradients in Sinusoidal Neural network","created":"2025-12-06","cited_by_count":1,"area":null},{"id":"2602.01734","title":"MSign: An Optimizer Preventing Training Instability in Large Language Models via Stable Rank Restoration","created":"2026-02-02","cited_by_count":1,"area":null},{"id":"2602.07494","title":"Hyperparameter Transfer Laws for Non-Recurrent Multi-Path Neural Networks","created":"2026-02-07","cited_by_count":1,"area":null},{"id":"2604.11890","title":"Subcritical Signal Propagation at Initialization in Normalization-Free Transformers","created":"2026-04-13","cited_by_count":1,"area":null},{"id":"2101.01178","title":"Advances in Electron Microscopy with Deep Learning","created":"2021-01-04","cited_by_count":0,"area":null},{"id":"2111.02155","title":"A Johnson--Lindenstrauss Framework for Randomly Initialized CNNs","created":"2021-11-03","cited_by_count":0,"area":null},{"id":"2210.02411","title":"Dynamical Isometry for Residual Networks","created":"2022-10-05","cited_by_count":0,"area":null},{"id":"2210.05268","title":"Component-Wise Natural Gradient Descent -- An Efficient Neural Network Optimization","created":"2022-10-11","cited_by_count":0,"area":null},{"id":"2402.09338","title":"Neural Networks Asymptotic Behaviours for the Resolution of Inverse Problems","created":"2024-02-14","cited_by_count":0,"area":null},{"id":"2411.19819","title":"GradAlign for Training-free Model Performance Inference","created":"2024-11-29","cited_by_count":0,"area":null},{"id":"2506.14386","title":"ResNets Are Deeper Than You Think","created":"2025-06-17","cited_by_count":0,"area":null},{"id":"2510.04327","title":"Arithmetic-Mean $\\mu$P for Modern Architectures: A Unified Learning-Rate Scale for CNNs and ResNets","created":"2025-10-05","cited_by_count":0,"area":null},{"id":"2510.08456","title":"Integral Signatures of Activation Functions: A 9-Dimensional Taxonomy and Stability Theory for Deep Learning","created":"2025-10-09","cited_by_count":0,"area":null},{"id":"2512.21075","title":"Feature Learning Dynamics in Infinite-Depth Neural Networks","created":"2025-12-24","cited_by_count":0,"area":null},{"id":"2602.20152","title":"Behavior Learning (BL): Learning Hierarchical Optimization Structures from Data","created":"2026-02-23","cited_by_count":0,"area":null},{"id":"2604.15742","title":"Collective Kernel EFT for Pre-activation ResNets","created":"2026-04-17","cited_by_count":0,"area":null},{"id":"2605.00662","title":"Spiking Sequence Machines and Transformers","created":"2026-05-01","cited_by_count":0,"area":null},{"id":"2605.13418","title":"DP-KFC: Data-Free Preconditioning for Privacy-Preserving Deep Learning","created":"2026-05-13","cited_by_count":0,"area":null},{"id":"2605.21648","title":"Dropout Universality: Scaling Laws and Optimal Scheduling at the Edge-of-Chaos","created":"2026-05-20","cited_by_count":0,"area":null},{"id":"2607.14233","title":"LIGO-PINN: Learned Initialization via Gated Optimization to Alleviate Convergence Failures in Physics Informed Neural Networks","created":"2026-07-15","cited_by_count":0,"area":null},{"id":"2609.09244","title":"Critical initialization destabilizes higher input derivatives in wide scalar-input networks","created":"2026-09-08","cited_by_count":0,"area":null},{"id":"2609.30633","title":"Stable initialization without the CLT","created":"2026-09-24","cited_by_count":0,"area":null}],"next_cursor":null}