{"total":121,"rows":[{"id":"2102.06810","title":"Understanding self-supervised Learning Dynamics without Contrastive Pairs","created":"2021-02-12","cited_by_count":256,"area":null},{"id":"2109.14545","title":"Activation Functions in Deep Learning: A Comprehensive Survey and Benchmark","created":"2021-09-29","cited_by_count":155,"area":null},{"id":"2103.10427","title":"The Low-Rank Simplicity Bias in Deep Networks","created":"2021-03-18","cited_by_count":154,"area":null},{"id":"2206.03126","title":"Signal Propagation in Transformers: Theoretical Perspectives and the Role of Rank Collapse","created":"2022-06-07","cited_by_count":141,"area":null},{"id":"2112.09153","title":"An Empirical Investigation of the Role of Pre-training in Lifelong Learning","created":"2021-12-16","cited_by_count":134,"area":null},{"id":"2104.07167","title":"Orthogonalizing Convolutional Layers with the Cayley Transform","created":"2021-04-14","cited_by_count":105,"area":null},{"id":"2107.02392","title":"Dirichlet Energy Constrained Learning for Deep Graph Neural Networks","created":"2021-07-06","cited_by_count":97,"area":null},{"id":"2107.11472","title":"Clipped Hyperbolic Classifiers Are Super-Hyperbolic Classifiers","created":"2021-07-23","cited_by_count":57,"area":null},{"id":"2409.14623","title":"From Lazy to Rich: Exact Learning Dynamics in Deep Linear Networks","created":"2024-09-22","cited_by_count":49,"area":null},{"id":"2502.10818","title":"On Vanishing Gradients, Over-Smoothing, and Over-Squashing in GNNs: Bridging Recurrent and Graph Learning","created":"2025-02-15","cited_by_count":43,"area":null},{"id":"2206.14476","title":"Can Push-forward Generative Models Fit Multimodal Distributions?","created":"2022-06-29","cited_by_count":41,"area":null},{"id":"2406.06811","title":"Learning Continually by Spectral Regularization","created":"2024-06-10","cited_by_count":34,"area":null},{"id":"2304.02840","title":"NTK-SAP: Improving neural network pruning by aligning training dynamics","created":"2023-04-06","cited_by_count":30,"area":null},{"id":"2110.12661","title":"ZerO Initialization: Initializing Neural Networks with only Zeros and Ones","created":"2021-10-25","cited_by_count":28,"area":null},{"id":"2203.08120","title":"Deep Learning without Shortcuts: Shaping the Kernel with Tailored Rectifiers","created":"2022-03-15","cited_by_count":28,"area":null},{"id":"2106.05956","title":"Beyond BatchNorm: Towards a Unified Understanding of Normalization in Deep Learning","created":"2021-06-10","cited_by_count":25,"area":null},{"id":"2212.13881","title":"Mechanism of feature learning in deep fully connected networks and kernel machines that recursively learn features","created":"2022-12-28","cited_by_count":25,"area":null},{"id":"2403.02579","title":"Geometric Dynamics of Signal Propagation Predict Trainability of Transformers","created":"2024-03-05","cited_by_count":24,"area":null},{"id":"2109.09304","title":"Deformed semicircle law and concentration of nonlinear random matrices for ultra-wide neural networks","created":"2021-09-20","cited_by_count":21,"area":null},{"id":"2307.06092","title":"Quantitative CLTs in Deep Neural Networks","created":"2023-07-12","cited_by_count":20,"area":null},{"id":"2509.10167","title":"The Hidden Width of Deep ResNets: Tight Error Bounds and Phase Diagram","created":"2025-09-12","cited_by_count":17,"area":null},{"id":"2405.15743","title":"Sparse maximal update parameterization: A holistic approach to sparse training dynamics","created":"2024-05-24","cited_by_count":16,"area":null},{"id":"2105.07741","title":"Activation function design for deep networks: linearity and effective initialisation","created":"2021-05-17","cited_by_count":13,"area":null},{"id":"2306.01485","title":"Robust low-rank training via approximate orthonormal constraints","created":"2023-06-02","cited_by_count":13,"area":null},{"id":"2306.12548","title":"Finite-time Lyapunov exponents of deep neural networks","created":"2023-06-21","cited_by_count":12,"area":null},{"id":"2401.10809","title":"Neglected Hessian component explains mysteries in Sharpness regularization","created":"2024-01-19","cited_by_count":12,"area":null},{"id":"2210.08122","title":"Old can be Gold: Better Gradient Flow can Make Vanilla-GCNs Great Again","created":"2022-10-14","cited_by_count":11,"area":null},{"id":"2310.06379","title":"Understanding the Expressivity and Trainability of Fourier Neural Operator: A Mean-Field Perspective","created":"2023-10-10","cited_by_count":11,"area":null},{"id":"2311.18718","title":"The Feature Speed Formula: a flexible approach to scale hyper-parameters of deep neural networks","created":"2023-11-30","cited_by_count":11,"area":null},{"id":"2405.19279","title":"Understanding and Minimising Outlier Features in Neural Network Training","created":"2024-05-29","cited_by_count":10,"area":null},{"id":"2203.11355","title":"Origami in N dimensions: How feed-forward networks manufacture linear separability","created":"2022-03-21","cited_by_count":9,"area":null},{"id":"2507.12709","title":"From SGD to Spectra: A Theory of Neural Network Weight Dynamics","created":"2025-07-17","cited_by_count":9,"area":null},{"id":"2207.09432","title":"Deep equilibrium networks are sensitive to initialization statistics","created":"2022-07-19","cited_by_count":8,"area":null},{"id":"2310.02012","title":"Towards Training Without Depth Limits: Batch Normalization Without Gradient Explosion","created":"2023-10-03","cited_by_count":8,"area":null},{"id":"2110.11477","title":"Conditioning of Random Feature Matrices: Double Descent and Generalization Error","created":"2021-10-21","cited_by_count":7,"area":null},{"id":"2210.05371","title":"On skip connections and normalisation layers in deep optimisation","created":"2022-10-10","cited_by_count":7,"area":null},{"id":"2602.08040","title":"FIRE: Frobenius-Isometry Reinitialization for Balancing the Stability-Plasticity Tradeoff","created":"2026-02-08","cited_by_count":7,"area":null},{"id":"2308.12585","title":"Persistent learning signals and working memory without continuous attractors","created":"2023-08-24","cited_by_count":6,"area":null},{"id":"2103.13466","title":"Asymptotic Freeness of Layerwise Jacobians Caused by Invariance of Multilayer Perceptron: The Haar Orthogonal Case","created":"2021-03-24","cited_by_count":5,"area":null},{"id":"2108.00051","title":"Coordinate descent on the orthogonal group for recurrent neural network training","created":"2021-07-30","cited_by_count":5,"area":null},{"id":"2109.08958","title":"AutoInit: Analytic Signal-Preserving Weight Initialization for Neural Networks","created":"2021-09-18","cited_by_count":5,"area":null},{"id":"2210.01245","title":"Random orthogonal additive filters: a solution to the vanishing/exploding gradient of deep neural networks","created":"2022-10-03","cited_by_count":5,"area":null},{"id":"2306.00809","title":"Initial Guessing Bias: How Untrained Networks Favor Some Classes","created":"2023-06-01","cited_by_count":5,"area":null},{"id":"2310.07765","title":"Feature Learning and Generalization in Deep Networks with Orthogonal Weights","created":"2023-10-11","cited_by_count":5,"area":null},{"id":"2312.17306","title":"Gradient Flossing: Improving Gradient Descent through Dynamic Control of Jacobians","created":"2023-12-28","cited_by_count":5,"area":null},{"id":"2402.16184","title":"Deep Neural Network Initialization with Sparsity Inducing Activations","created":"2024-02-25","cited_by_count":5,"area":null},{"id":"2503.04626","title":"IDInit: A Universal and Stable Initialization Method for Neural Network Training","created":"2025-03-06","cited_by_count":5,"area":null},{"id":"2505.19458","title":"Recurrent Self-Attention Dynamics: An Energy-Agnostic Perspective from Jacobians","created":"2025-05-26","cited_by_count":5,"area":null},{"id":"2602.18308","title":"JPmHC Dynamical Isometry via Orthogonal Hyper-Connections","created":"2026-02-20","cited_by_count":5,"area":null},{"id":"2605.14258","title":"Dynamics of the Transformer Residual Stream: Coupling Spectral Geometry to Network Topology","created":"2026-05-14","cited_by_count":5,"area":null},{"id":"2106.10704","title":"Better Training using Weight-Constrained Stochastic Dynamics","created":"2021-06-20","cited_by_count":4,"area":null},{"id":"2110.11150","title":"Lottery Tickets with Nonzero Biases","created":"2021-10-21","cited_by_count":4,"area":null},{"id":"2112.00980","title":"Trap of Feature Diversity in the Learning of MLPs","created":"2021-12-02","cited_by_count":4,"area":null},{"id":"2306.00700","title":"On the Weight Dynamics of Deep Normalized Networks","created":"2023-06-01","cited_by_count":4,"area":null},{"id":"2506.22908","title":"Attention to the Burstiness in Visual Prompt Tuning!","created":"2025-06-28","cited_by_count":4,"area":null},{"id":"2605.25880","title":"The Quantization Benefits of Residual-Free Transformers","created":"2026-05-25","cited_by_count":4,"area":null},{"id":"2201.11016","title":"Recency Dropout for Recurrent Recommender Systems","created":"2022-01-26","cited_by_count":3,"area":null},{"id":"2202.00834","title":"Nonlinear Initialization Methods for Low-Rank Neural Networks","created":"2022-02-02","cited_by_count":3,"area":null},{"id":"2204.04876","title":"Lyapunov-Guided Representation of Recurrent Neural Network Performance","created":"2022-04-11","cited_by_count":3,"area":null},{"id":"2406.01755","title":"Sparser, Better, Deeper, Stronger: Improving Sparse Training with Exact Orthogonal Initialization","created":"2024-06-03","cited_by_count":3,"area":null},{"id":"2506.08764","title":"On the Stability of the Jacobian Matrix in Deep Neural Networks","created":"2025-06-10","cited_by_count":3,"area":null},{"id":"2109.00125","title":"A Weight Initialization Based on the Linear Product Structure for Neural Networks","created":"2021-09-01","cited_by_count":2,"area":null},{"id":"2212.01744","title":"Statistical Physics of Deep Neural Networks: Initialization toward Optimal Channels","created":"2022-12-04","cited_by_count":2,"area":null},{"id":"2212.12086","title":"Eigenvalue initialisation and regularisation for Koopman autoencoders","created":"2022-12-23","cited_by_count":2,"area":null},{"id":"2304.06326","title":"Understanding Overfitting in Adversarial Training via Kernel Regression","created":"2023-04-13","cited_by_count":2,"area":null},{"id":"2310.07612","title":"PHYDI: Initializing Parameterized Hypercomplex Neural Networks as Identity Functions","created":"2023-10-11","cited_by_count":2,"area":null},{"id":"2505.11881","title":"Revisiting Residual Connections: Orthogonal Updates for Stable and Efficient Deep Networks","created":"2025-05-17","cited_by_count":2,"area":null},{"id":"2508.12121","title":"Time-Scale Coupling Between States and Parameters in Recurrent Neural Networks","created":"2025-08-16","cited_by_count":2,"area":null},{"id":"2510.04202","title":"Spectral Alignment as Predictor of Loss Explosion in Neural Network Training","created":"2025-10-05","cited_by_count":2,"area":null},{"id":"2602.03282","title":"Global Geometry Is Not Enough for Vision Representations","created":"2026-02-03","cited_by_count":2,"area":null},{"id":"2602.05779","title":"How Controlling the Variance can Improve Training Stability of Sparsely Activated DNNs and CNNs","created":"2026-02-05","cited_by_count":2,"area":null},{"id":"2606.20299","title":"Statistical Properties of Training & Generalization","created":"2026-06-18","cited_by_count":2,"area":null},{"id":"2606.28486","title":"Spectral phase transitions and trainability in neural network learning dynamics","created":"2026-06-26","cited_by_count":2,"area":null},{"id":"2607.14018","title":"Transforming Rank: How Architecture Navigates the Spectral Pathologies of Depth","created":"2026-07-15","cited_by_count":2,"area":null},{"id":"2204.07879","title":"Polynomial-time Sparse Measure Recovery: From Mean Field Theory to Algorithm Design","created":"2022-04-16","cited_by_count":1,"area":null},{"id":"2301.06956","title":"Expected Gradients of Maxout Networks and Consequences to Parameter Initialization","created":"2023-01-17","cited_by_count":1,"area":null},{"id":"2410.20107","title":"Emergence of Globally Attracting Fixed Points in Deep Neural Networks With Nonlinear Activations","created":"2024-10-26","cited_by_count":1,"area":null},{"id":"2510.06685","title":"Gaussian Equivalence for Self-Attention: Asymptotic Spectral Analysis of Attention Matrix","created":"2025-10-08","cited_by_count":1,"area":null},{"id":"2512.05084","title":"Gradient Descent with Provably Tuned Learning-rate Schedules","created":"2025-12-04","cited_by_count":1,"area":null},{"id":"2512.06427","title":"A new initialisation to Control Gradients in Sinusoidal Neural network","created":"2025-12-06","cited_by_count":1,"area":null},{"id":"2604.11890","title":"Subcritical Signal Propagation at Initialization in Normalization-Free Transformers","created":"2026-04-13","cited_by_count":1,"area":null},{"id":"2605.09861","title":"Flag Varieties: A Geometric Framework for Deep Network Alignment","created":"2026-05-11","cited_by_count":1,"area":null},{"id":"2606.09762","title":"Preserving Plasticity in Continual Learning via Dynamical Isometry","created":"2026-06-08","cited_by_count":1,"area":null},{"id":"2111.02155","title":"A Johnson--Lindenstrauss Framework for Randomly Initialized CNNs","created":"2021-11-03","cited_by_count":0,"area":null},{"id":"2201.07934","title":"Complexity from Adaptive-Symmetries Breaking: Global Minima in the Statistical Mechanics of Deep Neural Networks","created":"2022-01-03","cited_by_count":0,"area":null},{"id":"2210.02411","title":"Dynamical Isometry for Residual Networks","created":"2022-10-05","cited_by_count":0,"area":null},{"id":"2307.02263","title":"Dynamical Isometry based Rigorous Fair Neural Architecture Search","created":"2023-07-05","cited_by_count":0,"area":null},{"id":"2310.14206","title":"Manifold-Preserving Transformers are Effective for Short-Long Range Encoding","created":"2023-10-22","cited_by_count":0,"area":null},{"id":"2505.03646","title":"Revealing Hidden Vulnerabilities in Autoencoders through Gradient Signal Restoration","created":"2025-05-06","cited_by_count":0,"area":null},{"id":"2506.14386","title":"ResNets Are Deeper Than You Think","created":"2025-06-17","cited_by_count":0,"area":null},{"id":"2507.07625","title":"Concentration of measure for non-linear random matrices with applications to neural networks and non-commutative polynomials","created":"2025-07-10","cited_by_count":0,"area":null},{"id":"2508.02882","title":"Deep Network Trainability via Persistent Subspace Orthogonality","created":"2025-08-04","cited_by_count":0,"area":null},{"id":"2509.00362","title":"Optimized Weight Initialization on the Stiefel Manifold for Deep ReLU Neural Networks","created":"2025-08-30","cited_by_count":0,"area":null},{"id":"2510.08456","title":"Integral Signatures of Activation Functions: A 9-Dimensional Taxonomy and Stability Theory for Deep Learning","created":"2025-10-09","cited_by_count":0,"area":null},{"id":"2512.05790","title":"Learnability Window in Gated Recurrent Neural Networks","created":"2025-12-05","cited_by_count":0,"area":null},{"id":"2512.12617","title":"Spectral Sentinel: Scalable Byzantine-Robust Decentralized Federated Learning via Sketched Random Matrix Theory on Blockchain","created":"2025-12-14","cited_by_count":0,"area":null},{"id":"2601.06441","title":"FlexAct: Why Learn when you can Pick?","created":"2026-01-10","cited_by_count":0,"area":null},{"id":"2602.00130","title":"On the Relationship Between Representation Geometry and Generalization in Deep Neural Networks","created":"2026-01-28","cited_by_count":0,"area":null},{"id":"2602.01136","title":"A Unified Matrix-Spectral Framework for Stability and Interpretability in Deep Learning","created":"2026-02-01","cited_by_count":0,"area":null},{"id":"2602.08287","title":"Noise Stability of Transformer Models","created":"2026-02-09","cited_by_count":0,"area":null}],"next_cursor":"100"}