{"total":261,"rows":[{"id":"2110.01889","title":"Deep Neural Networks and Tabular Data: A Survey","created":"2021-10-05","cited_by_count":491,"area":null},{"id":"2108.08481","title":"Neural Operator: Learning Maps Between Function Spaces","created":"2021-08-19","cited_by_count":414,"area":null},{"id":"2303.01486","title":"Understanding plasticity in neural networks","created":"2023-03-02","cited_by_count":174,"area":null},{"id":"2206.03126","title":"Signal Propagation in Transformers: Theoretical Perspectives and the Role of Rank Collapse","created":"2022-06-07","cited_by_count":141,"area":null},{"id":"2310.17813","title":"A Spectral Condition for Feature Learning","created":"2023-10-26","cited_by_count":109,"area":null},{"id":"2205.09653","title":"Self-Consistent Dynamical Field Theory of Kernel Evolution in Wide Neural Networks","created":"2022-05-19","cited_by_count":105,"area":null},{"id":"2402.18762","title":"Disentangling the Causes of Plasticity Loss in Neural Networks","created":"2024-02-29","cited_by_count":86,"area":"deeprl"},{"id":"2505.01618","title":"Don't be lazy: CompleteP enables compute-efficient deep transformers","created":"2025-05-02","cited_by_count":78,"area":null},{"id":"2307.11411","title":"Deep Directly-Trained Spiking Neural Networks for Object Detection","created":"2023-07-21","cited_by_count":77,"area":null},{"id":"2102.01063","title":"Zen-NAS: A Zero-Shot NAS for High-Performance Deep Image Recognition","created":"2021-02-01","cited_by_count":73,"area":null},{"id":"2512.24695","title":"Nested Learning: The Illusion of Deep Learning Architectures","created":"2025-12-31","cited_by_count":72,"area":null},{"id":"2103.05127","title":"Model Complexity of Deep Learning: A Survey","created":"2021-03-08","cited_by_count":65,"area":null},{"id":"2407.01800","title":"Normalization and effective learning rates in reinforcement learning","created":"2024-07-01","cited_by_count":58,"area":"deeprl"},{"id":"2306.17759","title":"The Shaped Transformer: Attention Models in the Infinite Depth-and-Width Limit","created":"2023-06-30","cited_by_count":47,"area":null},{"id":"2406.09405","title":"Why Warmup the Learning Rate? Underlying Mechanisms and Improvements","created":"2024-06-13","cited_by_count":46,"area":null},{"id":"2302.10322","title":"Deep Transformers without Shortcuts: Modifying Self-attention for Faithful Signal Propagation","created":"2023-02-20","cited_by_count":45,"area":null},{"id":"2403.02241","title":"Neural Redshift: Random Networks are not Random Functions","created":"2024-03-04","cited_by_count":43,"area":null},{"id":"2502.10818","title":"On Vanishing Gradients, Over-Smoothing, and Over-Squashing in GNNs: Bridging Recurrent and Graph Learning","created":"2025-02-15","cited_by_count":43,"area":null},{"id":"2306.13575","title":"Scaling MLPs: A Tale of Inductive Bias","created":"2023-06-23","cited_by_count":37,"area":null},{"id":"2311.01906","title":"Simplifying Transformer Blocks","created":"2023-11-03","cited_by_count":37,"area":null},{"id":"2202.00553","title":"Neural Tangent Kernel Beyond the Infinite-Width Limit: Effects of Depth and Initialization","created":"2022-02-01","cited_by_count":30,"area":null},{"id":"2604.21691","title":"There Will Be a Scientific Theory of Deep Learning","created":"2026-04-23","cited_by_count":29,"area":null},{"id":"2103.04992","title":"Self-learning Machines based on Hamiltonian Echo Backpropagation","created":"2021-03-08","cited_by_count":28,"area":null},{"id":"2104.11044","title":"Analyzing Monotonic Linear Interpolation in Neural Network Loss Landscapes","created":"2021-04-22","cited_by_count":28,"area":null},{"id":"2106.06615","title":"Precise characterization of the prior predictive distribution of deep ReLU networks","created":"2021-06-11","cited_by_count":28,"area":null},{"id":"2203.08120","title":"Deep Learning without Shortcuts: Shaping the Kernel with Tailored Rectifiers","created":"2022-03-15","cited_by_count":28,"area":null},{"id":"2304.06670","title":"Deep neural networks have an inbuilt Occam's razor","created":"2023-04-13","cited_by_count":28,"area":null},{"id":"2309.16314","title":"A Primer on Bayesian Neural Networks: Review and Debates","created":"2023-09-28","cited_by_count":27,"area":null},{"id":"2207.14293","title":"Image sensing with multilayer, nonlinear optical neural networks","created":"2022-07-27","cited_by_count":25,"area":null},{"id":"2303.17671","title":"Neural signature kernels as infinite-width-depth-limits of controlled ResNets","created":"2023-03-30","cited_by_count":25,"area":null},{"id":"2109.13247","title":"The edge of chaos: quantum field theory and deep neural networks","created":"2021-09-27","cited_by_count":24,"area":null},{"id":"2403.02579","title":"Geometric Dynamics of Signal Propagation Predict Trainability of Transformers","created":"2024-03-05","cited_by_count":24,"area":null},{"id":"2310.07999","title":"LEMON: Lossless model expansion","created":"2023-10-12","cited_by_count":23,"area":"lr"},{"id":"2210.00688","title":"On the infinite-depth limit of finite-width neural networks","created":"2022-10-03","cited_by_count":22,"area":null},{"id":"2211.07844","title":"Characterizing the Spectrum of the NTK via a Power Series Expansion","created":"2022-11-15","cited_by_count":22,"area":null},{"id":"2311.02076","title":"Universal Sharpness Dynamics in Neural Network Training: Fixed Point Analysis, Edge of Stability, and Route to Chaos","created":"2023-11-03","cited_by_count":22,"area":null},{"id":"2311.04128","title":"Generative learning for nonlinear dynamics","created":"2023-11-07","cited_by_count":22,"area":null},{"id":"2106.06529","title":"The Limitations of Large Width in Neural Networks: A Deep Gaussian Process Perspective","created":"2021-06-11","cited_by_count":21,"area":null},{"id":"2109.09304","title":"Deformed semicircle law and concentration of nonlinear random matrices for ultra-wide neural networks","created":"2021-09-20","cited_by_count":21,"area":null},{"id":"2302.00453","title":"Width and Depth Limits Commute in Residual Networks","created":"2023-02-01","cited_by_count":21,"area":null},{"id":"2304.02034","title":"Effective Theory of Transformers at Initialization","created":"2023-04-04","cited_by_count":21,"area":null},{"id":"2307.06092","title":"Quantitative CLTs in Deep Neural Networks","created":"2023-07-12","cited_by_count":20,"area":null},{"id":"2401.07187","title":"A Survey on Statistical Theory of Deep Learning: Approximation, Training Dynamics, and Generative Models","created":"2024-01-14","cited_by_count":20,"area":null},{"id":"2210.04909","title":"Meta-Principled Family of Hyperparameter Scaling Strategies","created":"2022-10-10","cited_by_count":19,"area":null},{"id":"2302.12250","title":"Phase diagram of early training dynamics in deep neural networks: effect of the learning rate, depth, and width","created":"2023-02-23","cited_by_count":19,"area":null},{"id":"2305.06986","title":"Provable Guarantees for Nonlinear Feature Learning in Three-Layer Neural Networks","created":"2023-05-11","cited_by_count":19,"area":null},{"id":"2102.01621","title":"Depth separation beyond radial functions","created":"2021-02-02","cited_by_count":18,"area":null},{"id":"2110.11749","title":"Feature Learning and Signal Propagation in Deep Neural Networks","created":"2021-10-22","cited_by_count":18,"area":null},{"id":"2512.01643","title":"ViT$^3$: Unlocking Test-Time Training in Vision","created":"2025-12-01","cited_by_count":18,"area":null},{"id":"2102.10492","title":"Deep ReLU Networks Preserve Expected Length","created":"2021-02-21","cited_by_count":17,"area":null},{"id":"2112.02393","title":"Optimization-Based Separations for Neural Networks","created":"2021-12-04","cited_by_count":17,"area":null},{"id":"2112.05611","title":"Eigenspace Restructuring: a Principle of Space and Frequency in Neural Networks","created":"2021-12-10","cited_by_count":17,"area":null},{"id":"2505.24333","title":"Two failure modes of deep transformers and how to avoid them: a unified theory of signal propagation at initialisation","created":"2025-05-30","cited_by_count":17,"area":null},{"id":"2111.13336","title":"MAE-DET: Revisiting Maximum Entropy Principle in Zero-Shot NAS for Efficient Object Detection","created":"2021-11-26","cited_by_count":16,"area":null},{"id":"2107.12530","title":"Convergence of Deep ReLU Networks","created":"2021-07-27","cited_by_count":15,"area":null},{"id":"2202.11921","title":"Auto-scaling Vision Transformers without Training","created":"2022-02-24","cited_by_count":15,"area":null},{"id":"2303.00579","title":"Are More Layers Beneficial to Graph Transformers?","created":"2023-03-01","cited_by_count":15,"area":null},{"id":"2103.03113","title":"Towards Deepening Graph Neural Networks: A GNTK-based Optimization Perspective","created":"2021-03-03","cited_by_count":14,"area":null},{"id":"2107.06898","title":"Towards quantifying information flows: relative entropy in deep neural networks and the renormalization group","created":"2021-07-14","cited_by_count":14,"area":null},{"id":"2310.10767","title":"Wide Neural Networks as Gaussian Processes: Lessons from Deep Equilibrium Models","created":"2023-10-16","cited_by_count":14,"area":null},{"id":"2105.07741","title":"Activation function design for deep networks: linearity and effective initialisation","created":"2021-05-17","cited_by_count":13,"area":null},{"id":"2305.02334","title":"Structures of Neural Network Effective Theories","created":"2023-05-03","cited_by_count":13,"area":null},{"id":"2305.18399","title":"On the impact of activation and normalization in obtaining isometric embeddings at initialization","created":"2023-05-28","cited_by_count":13,"area":null},{"id":"2106.03186","title":"Reverse Engineering the Neural Tangent Kernel","created":"2021-06-06","cited_by_count":12,"area":null},{"id":"2202.04925","title":"Decomposing neural networks as mappings of correlation functions","created":"2022-02-10","cited_by_count":12,"area":null},{"id":"2204.14126","title":"Wide and Deep Neural Networks Achieve Optimality for Classification","created":"2022-04-29","cited_by_count":12,"area":null},{"id":"2205.08099","title":"Dimensionality Reduced Training by Pruning and Freezing Parts of a Deep Neural Network, a Survey","created":"2022-05-17","cited_by_count":12,"area":null},{"id":"2301.00008","title":"Effects of Data Geometry in Early Deep Learning","created":"2022-12-29","cited_by_count":12,"area":null},{"id":"2306.12548","title":"Finite-time Lyapunov exponents of deep neural networks","created":"2023-06-21","cited_by_count":12,"area":null},{"id":"2312.12474","title":"Principled Weight Initialisation for Input-Convex Neural Networks","created":"2023-12-19","cited_by_count":12,"area":null},{"id":"2506.01115","title":"Is Random Attention Sufficient for Sequence Modeling? Disentangling Trainable Components in the Transformer","created":"2025-06-01","cited_by_count":12,"area":"lr"},{"id":"2305.19730","title":"Data Representations' Study of Latent Image Manifolds","created":"2023-05-31","cited_by_count":11,"area":null},{"id":"2310.06379","title":"Understanding the Expressivity and Trainability of Fourier Neural Operator: A Mean-Field Perspective","created":"2023-10-10","cited_by_count":11,"area":null},{"id":"2311.18718","title":"The Feature Speed Formula: a flexible approach to scale hyper-parameters of deep neural networks","created":"2023-11-30","cited_by_count":11,"area":null},{"id":"2107.09437","title":"Edge of chaos as a guiding principle for modern neural network training","created":"2021-07-20","cited_by_count":10,"area":null},{"id":"2402.09142","title":"When Representations Align: Universality in Representation Learning Dynamics","created":"2024-02-14","cited_by_count":10,"area":null},{"id":"2405.19279","title":"Understanding and Minimising Outlier Features in Neural Network Training","created":"2024-05-29","cited_by_count":10,"area":null},{"id":"2502.03210","title":"From Kernels to Features: A Multi-Scale Adaptive Theory of Feature Learning","created":"2025-02-05","cited_by_count":10,"area":null},{"id":"2205.12379","title":"Gaussian Pre-Activations in Neural Networks: Myth or Reality?","created":"2022-05-24","cited_by_count":9,"area":null},{"id":"2304.01063","title":"Depth Separation with Multilayer Mean-Field Networks","created":"2023-04-03","cited_by_count":9,"area":null},{"id":"2405.17538","title":"Bayesian RG Flow in Neural Network Field Theories","created":"2024-05-27","cited_by_count":9,"area":null},{"id":"2108.02316","title":"Deep Stable neural networks: large-width asymptotics and convergence rates","created":"2021-08-02","cited_by_count":8,"area":null},{"id":"2111.12143","title":"Critical Initialization of Wide and Deep Neural Networks through Partial Jacobians: General Theory and Applications","created":"2021-11-23","cited_by_count":8,"area":null},{"id":"2207.09432","title":"Deep equilibrium networks are sensitive to initialization statistics","created":"2022-07-19","cited_by_count":8,"area":null},{"id":"2210.09809","title":"Analysis of Convolutions, Non-linearity and Depth in Graph Neural Networks using Neural Tangent Kernel","created":"2022-10-18","cited_by_count":8,"area":null},{"id":"2405.18979","title":"MANO: Exploiting Matrix Norm for Unsupervised Accuracy Estimation Under Distribution Shifts","created":"2024-05-29","cited_by_count":8,"area":null},{"id":"2505.22785","title":"Navigating the Latent Space Dynamics of Neural Models","created":"2025-05-28","cited_by_count":8,"area":null},{"id":"2507.20057","title":"What Can Grokking Teach Us About Learning Under Nonstationarity?","created":"2025-07-26","cited_by_count":8,"area":null},{"id":"2104.00008","title":"Why is AI hard and Physics simple?","created":"2021-03-31","cited_by_count":7,"area":null},{"id":"2106.03091","title":"Regularization in ResNet with Stochastic Depth","created":"2021-06-06","cited_by_count":7,"area":null},{"id":"2208.01134","title":"Improving the Trainability of Deep Neural Networks through Layerwise Batch-Entropy Regularization","created":"2022-08-01","cited_by_count":7,"area":null},{"id":"2211.06077","title":"Overparameterized random feature regression with nearly orthogonal data","created":"2022-11-11","cited_by_count":7,"area":null},{"id":"2310.14062","title":"On the Neural Tangent Kernel of Equilibrium Models","created":"2023-10-21","cited_by_count":7,"area":null},{"id":"2405.17088","title":"Phase Transitions in the Output Distribution of Large Language Models","created":"2024-05-27","cited_by_count":7,"area":null},{"id":"2406.02269","title":"Graph Neural Networks Do Not Always Oversmooth","created":"2024-06-04","cited_by_count":7,"area":null},{"id":"2408.08307","title":"What Secrets Do Your Manifolds Hold? Understanding the Local Geometry of Generative Models","created":"2024-08-15","cited_by_count":7,"area":"diffusion"},{"id":"2602.10959","title":"Rotary Positional Embeddings as Phase Modulation: Theoretical Bounds on the RoPE Base for Long-Context Transformers","created":"2026-02-11","cited_by_count":7,"area":null},{"id":"2207.04612","title":"Synergy and Symmetry in Deep Learning: Interactions between the Data, Model, and Inference Algorithm","created":"2022-07-11","cited_by_count":6,"area":null},{"id":"2301.09245","title":"Towards NeuroAI: Introducing Neuronal Diversity into Artificial Neural Networks","created":"2023-01-23","cited_by_count":6,"area":null},{"id":"2303.05161","title":"Inversion dynamics of class manifolds in deep learning reveals tradeoffs underlying generalisation","created":"2023-03-09","cited_by_count":6,"area":null}],"next_cursor":"100"}