{"total":166,"rows":[{"id":"2103.17239","title":"Going deeper with Image Transformers","created":"2021-03-31","cited_by_count":764,"area":null},{"id":"2101.03697","title":"RepVGG: Making VGG-style ConvNets Great Again","created":"2021-01-11","cited_by_count":522,"area":null},{"id":"2102.06701","title":"Explaining Neural Scaling Laws","created":"2021-02-12","cited_by_count":363,"area":null},{"id":"2105.02375","title":"A Geometric Analysis of Neural Collapse with Unconstrained Features","created":"2021-05-06","cited_by_count":227,"area":null},{"id":"2307.01694","title":"Spike-driven Transformer","created":"2023-07-04","cited_by_count":146,"area":null},{"id":"2206.03126","title":"Signal Propagation in Transformers: Theoretical Perspectives and the Role of Rank Collapse","created":"2022-06-07","cited_by_count":141,"area":null},{"id":"2112.09153","title":"An Empirical Investigation of the Role of Pre-training in Lifelong Learning","created":"2021-12-16","cited_by_count":134,"area":null},{"id":"2405.00781","title":"Barren Plateaus in Variational Quantum Computing","created":"2024-05-01","cited_by_count":133,"area":null},{"id":"2206.00133","title":"Pre-training via Denoising for Molecular Property Prediction","created":"2022-05-31","cited_by_count":124,"area":null},{"id":"2106.11342","title":"Dive into Deep Learning","created":"2021-06-21","cited_by_count":111,"area":null},{"id":"2104.07167","title":"Orthogonalizing Convolutional Layers with the Cayley Transform","created":"2021-04-14","cited_by_count":105,"area":null},{"id":"2112.08954","title":"Advancing Spiking Neural Networks towards Deep Residual Learning","created":"2021-12-15","cited_by_count":79,"area":null},{"id":"2505.01618","title":"Don't be lazy: CompleteP enables compute-efficient deep transformers","created":"2025-05-02","cited_by_count":78,"area":null},{"id":"2105.11417","title":"Skew Orthogonal Convolutions","created":"2021-05-24","cited_by_count":67,"area":null},{"id":"2206.08720","title":"Fast Finite Width Neural Tangent Kernel","created":"2022-06-17","cited_by_count":62,"area":null},{"id":"2110.12690","title":"A Dynamical System Perspective for Lipschitz Neural Networks","created":"2021-10-25","cited_by_count":60,"area":null},{"id":"2108.04062","title":"Improved deterministic l2 robustness on CIFAR-10 and CIFAR-100","created":"2021-08-05","cited_by_count":47,"area":null},{"id":"2206.06072","title":"Rank Diminishing in Deep Neural Networks","created":"2022-06-13","cited_by_count":47,"area":null},{"id":"2302.10322","title":"Deep Transformers without Shortcuts: Modifying Self-attention for Faithful Signal Propagation","created":"2023-02-20","cited_by_count":45,"area":null},{"id":"2311.01906","title":"Simplifying Transformer Blocks","created":"2023-11-03","cited_by_count":37,"area":null},{"id":"2406.06811","title":"Learning Continually by Spectral Regularization","created":"2024-06-10","cited_by_count":34,"area":null},{"id":"2102.10032","title":"Approximation and Learning with Deep Convolutional Models: a Kernel Perspective","created":"2021-02-19","cited_by_count":31,"area":null},{"id":"2106.05275","title":"Tractable Density Estimation on Learned Manifolds with Conformal Embedding Flows","created":"2021-06-09","cited_by_count":31,"area":null},{"id":"2206.12543","title":"A Fast, Well-Founded Approximation to the Empirical Neural Tangent Kernel","created":"2022-06-25","cited_by_count":30,"area":null},{"id":"2110.12661","title":"ZerO Initialization: Initializing Neural Networks with only Zeros and Ones","created":"2021-10-25","cited_by_count":28,"area":null},{"id":"2203.08120","title":"Deep Learning without Shortcuts: Shaping the Kernel with Tailored Rectifiers","created":"2022-03-15","cited_by_count":28,"area":null},{"id":"2203.05483","title":"projUNN: efficient method for training deep networks with unitary matrices","created":"2022-03-10","cited_by_count":27,"area":null},{"id":"2212.13881","title":"Mechanism of feature learning in deep fully connected networks and kernel machines that recursively learn features","created":"2022-12-28","cited_by_count":25,"area":null},{"id":"2109.13247","title":"The edge of chaos: quantum field theory and deep neural networks","created":"2021-09-27","cited_by_count":24,"area":null},{"id":"2208.13614","title":"Neural Tangent Kernel: A Survey","created":"2022-08-29","cited_by_count":23,"area":null},{"id":"2109.09304","title":"Deformed semicircle law and concentration of nonlinear random matrices for ultra-wide neural networks","created":"2021-09-20","cited_by_count":21,"area":null},{"id":"2306.01154","title":"The Law of Parsimony in Gradient Descent for Learning Deep Linear Networks","created":"2023-06-01","cited_by_count":20,"area":null},{"id":"2109.11338","title":"Orthogonal Graph Neural Networks","created":"2021-09-23","cited_by_count":19,"area":null},{"id":"2305.06986","title":"Provable Guarantees for Nonlinear Feature Learning in Three-Layer Neural Networks","created":"2023-05-11","cited_by_count":19,"area":null},{"id":"2110.11749","title":"Feature Learning and Signal Propagation in Deep Neural Networks","created":"2021-10-22","cited_by_count":18,"area":null},{"id":"2112.05611","title":"Eigenspace Restructuring: a Principle of Space and Frequency in Neural Networks","created":"2021-12-10","cited_by_count":17,"area":null},{"id":"2505.24333","title":"Two failure modes of deep transformers and how to avoid them: a unified theory of signal propagation at initialisation","created":"2025-05-30","cited_by_count":17,"area":null},{"id":"2108.05623","title":"Existence, Stability and Scalability of Orthogonal Convolutional Neural Networks","created":"2021-08-12","cited_by_count":16,"area":null},{"id":"2210.03651","title":"Understanding the Covariance Structure of Convolutional Filters","created":"2022-10-07","cited_by_count":16,"area":null},{"id":"2301.05219","title":"Why is the State of Neural Network Pruning so Confusing? On the Fairness, Comparison Setup, and Trainability in Network Pruning","created":"2023-01-12","cited_by_count":16,"area":null},{"id":"2311.16833","title":"1-Lipschitz Layers Compared: Memory, Speed, and Certifiable Robustness","created":"2023-11-28","cited_by_count":16,"area":null},{"id":"2103.00673","title":"Convolutional Normalization: Improving Deep Convolutional Network Robustness and Training","created":"2021-03-01","cited_by_count":15,"area":null},{"id":"2303.00579","title":"Are More Layers Beneficial to Graph Transformers?","created":"2023-03-01","cited_by_count":15,"area":null},{"id":"2312.12226","title":"On the Parameterization of Second-Order Optimization Effective Towards the Infinite Width","created":"2023-12-19","cited_by_count":15,"area":null},{"id":"2107.06898","title":"Towards quantifying information flows: relative entropy in deep neural networks and the renormalization group","created":"2021-07-14","cited_by_count":14,"area":null},{"id":"2305.18399","title":"On the impact of activation and normalization in obtaining isometric embeddings at initialization","created":"2023-05-28","cited_by_count":13,"area":null},{"id":"2306.01485","title":"Robust low-rank training via approximate orthonormal constraints","created":"2023-06-02","cited_by_count":13,"area":null},{"id":"2204.14126","title":"Wide and Deep Neural Networks Achieve Optimality for Classification","created":"2022-04-29","cited_by_count":12,"area":null},{"id":"2205.08099","title":"Dimensionality Reduced Training by Pruning and Freezing Parts of a Deep Neural Network, a Survey","created":"2022-05-17","cited_by_count":12,"area":null},{"id":"2312.12474","title":"Principled Weight Initialisation for Input-Convex Neural Networks","created":"2023-12-19","cited_by_count":12,"area":null},{"id":"2205.14173","title":"Momentum Stiefel Optimizer, with Applications to Suitably-Orthogonal Attention, and Optimal Transport","created":"2022-05-27","cited_by_count":11,"area":null},{"id":"2310.06379","title":"Understanding the Expressivity and Trainability of Fourier Neural Operator: A Mean-Field Perspective","created":"2023-10-10","cited_by_count":11,"area":null},{"id":"2410.04779","title":"Fast Training of Sinusoidal Neural Fields via Scaling Initialization","created":"2024-10-07","cited_by_count":11,"area":null},{"id":"2210.02373","title":"Dynamical systems' based neural networks","created":"2022-10-05","cited_by_count":10,"area":null},{"id":"2402.09142","title":"When Representations Align: Universality in Representation Learning Dynamics","created":"2024-02-14","cited_by_count":10,"area":null},{"id":"2405.19279","title":"Understanding and Minimising Outlier Features in Neural Network Training","created":"2024-05-29","cited_by_count":10,"area":null},{"id":"2502.03210","title":"From Kernels to Features: A Multi-Scale Adaptive Theory of Feature Learning","created":"2025-02-05","cited_by_count":10,"area":null},{"id":"2102.07655","title":"Dense for the Price of Sparse: Improved Performance of Sparsely Initialized Networks via a Subspace Offset","created":"2021-02-12","cited_by_count":9,"area":null},{"id":"2109.12960","title":"Ridgeless Interpolation with Shallow ReLU Networks in $1D$ is Nearest Neighbor Curvature Extrapolation and Provably Generalizes on Lipschitz Functions","created":"2021-09-27","cited_by_count":9,"area":null},{"id":"2112.09831","title":"Neural Born Iteration Method For Solving Inverse Scattering Problems: 2D Cases","created":"2021-12-18","cited_by_count":9,"area":null},{"id":"2203.11355","title":"Origami in N dimensions: How feed-forward networks manufacture linear separability","created":"2022-03-21","cited_by_count":9,"area":null},{"id":"2211.08453","title":"Improved techniques for deterministic l2 robustness","created":"2022-11-15","cited_by_count":9,"area":null},{"id":"2111.12143","title":"Critical Initialization of Wide and Deep Neural Networks through Partial Jacobians: General Theory and Applications","created":"2021-11-23","cited_by_count":8,"area":null},{"id":"2207.09432","title":"Deep equilibrium networks are sensitive to initialization statistics","created":"2022-07-19","cited_by_count":8,"area":null},{"id":"2209.12466","title":"Learned Force Fields Are Ready For Ground State Catalyst Discovery","created":"2022-09-26","cited_by_count":8,"area":null},{"id":"2210.09809","title":"Analysis of Convolutions, Non-linearity and Depth in Graph Neural Networks using Neural Tangent Kernel","created":"2022-10-18","cited_by_count":8,"area":null},{"id":"2310.02012","title":"Towards Training Without Depth Limits: Batch Normalization Without Gradient Explosion","created":"2023-10-03","cited_by_count":8,"area":null},{"id":"2410.05499","title":"Unitary convolutions for learning on graphs and groups","created":"2024-10-07","cited_by_count":8,"area":null},{"id":"2501.07930","title":"An Adaptive Orthogonal Convolution Scheme for Efficient and Flexible CNN Architectures","created":"2025-01-14","cited_by_count":8,"area":null},{"id":"2102.03793","title":"Tilting the playing field: Dynamical loss functions for machine learning","created":"2021-02-07","cited_by_count":7,"area":null},{"id":"2106.03091","title":"Regularization in ResNet with Stochastic Depth","created":"2021-06-06","cited_by_count":7,"area":null},{"id":"2106.09121","title":"Scaling-up Diverse Orthogonal Convolutional Networks with a Paraunitary Framework","created":"2021-06-16","cited_by_count":7,"area":null},{"id":"2208.01134","title":"Improving the Trainability of Deep Neural Networks through Layerwise Batch-Entropy Regularization","created":"2022-08-01","cited_by_count":7,"area":null},{"id":"2210.05371","title":"On skip connections and normalisation layers in deep optimisation","created":"2022-10-10","cited_by_count":7,"area":null},{"id":"2310.14062","title":"On the Neural Tangent Kernel of Equilibrium Models","created":"2023-10-21","cited_by_count":7,"area":null},{"id":"2409.11859","title":"Tight and Efficient Upper Bound on Spectral Norm of Convolutional Layers","created":"2024-09-18","cited_by_count":7,"area":null},{"id":"2505.12003","title":"Approximation theory for 1-Lipschitz ResNets","created":"2025-05-17","cited_by_count":7,"area":null},{"id":"2602.08040","title":"FIRE: Frobenius-Isometry Reinitialization for Balancing the Stability-Plasticity Tradeoff","created":"2026-02-08","cited_by_count":7,"area":null},{"id":"2207.04612","title":"Synergy and Symmetry in Deep Learning: Interactions between the Data, Model, and Inference Algorithm","created":"2022-07-11","cited_by_count":6,"area":null},{"id":"2306.03335","title":"Unraveling Projection Heads in Contrastive Learning: Insights from Expansion and Shrinkage","created":"2023-06-06","cited_by_count":6,"area":null},{"id":"2603.11970","title":"Statistical and structural identifiability in representation learning","created":"2026-03-12","cited_by_count":6,"area":null},{"id":"2103.13466","title":"Asymptotic Freeness of Layerwise Jacobians Caused by Invariance of Multilayer Perceptron: The Haar Orthogonal Case","created":"2021-03-24","cited_by_count":5,"area":null},{"id":"2109.08958","title":"AutoInit: Analytic Signal-Preserving Weight Initialization for Neural Networks","created":"2021-09-18","cited_by_count":5,"area":null},{"id":"2111.00687","title":"RMNet: Equivalently Removing Residual Connection from Networks","created":"2021-11-01","cited_by_count":5,"area":null},{"id":"2205.08385","title":"Feedback Gradient Descent: Efficient and Stable Optimization with Orthogonality for DNNs","created":"2022-05-12","cited_by_count":5,"area":null},{"id":"2208.06028","title":"Gaussian Process Surrogate Models for Neural Networks","created":"2022-08-11","cited_by_count":5,"area":null},{"id":"2210.01245","title":"Random orthogonal additive filters: a solution to the vanishing/exploding gradient of deep neural networks","created":"2022-10-03","cited_by_count":5,"area":null},{"id":"2210.01506","title":"How deep convolutional neural networks lose spatial information with training","created":"2022-10-04","cited_by_count":5,"area":null},{"id":"2212.05599","title":"Orthogonal SVD Covariance Conditioning and Latent Disentanglement","created":"2022-12-11","cited_by_count":5,"area":null},{"id":"2301.13710","title":"On the Initialisation of Wide Low-Rank Feedforward Neural Networks","created":"2023-01-31","cited_by_count":5,"area":null},{"id":"2310.07765","title":"Feature Learning and Generalization in Deep Networks with Orthogonal Weights","created":"2023-10-11","cited_by_count":5,"area":null},{"id":"2402.16184","title":"Deep Neural Network Initialization with Sparsity Inducing Activations","created":"2024-02-25","cited_by_count":5,"area":null},{"id":"2505.15174","title":"Enhancing Certified Robustness via Block Reflector Orthogonal Layers and Logit Annealing Loss","created":"2025-05-21","cited_by_count":5,"area":null},{"id":"2104.05878","title":"On the validity of kernel approximations for orthogonally-initialized neural networks","created":"2021-04-13","cited_by_count":4,"area":null},{"id":"2106.04741","title":"Marginalizable Density Models","created":"2021-06-08","cited_by_count":4,"area":null},{"id":"2106.10704","title":"Better Training using Weight-Constrained Stochastic Dynamics","created":"2021-06-20","cited_by_count":4,"area":null},{"id":"2207.02119","title":"Improving Covariance Conditioning of the SVD Meta-layer by Orthogonality","created":"2022-07-05","cited_by_count":4,"area":null},{"id":"2307.02284","title":"Universal Scaling Laws of Absorbing Phase Transitions in Artificial Deep Neural Networks","created":"2023-07-05","cited_by_count":4,"area":null},{"id":"2405.16639","title":"A direct proof of a unified law of robustness for Bregman divergence losses","created":"2024-05-26","cited_by_count":4,"area":null},{"id":"2406.06504","title":"Equivariant Neural Tangent Kernels","created":"2024-06-10","cited_by_count":4,"area":null}],"next_cursor":"100"}