{"total":173,"rows":[{"id":"2202.10054","title":"Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution","created":"2022-02-21","cited_by_count":739,"area":null},{"id":"2306.09927","title":"Trained Transformers Learn Linear Models In-Context","created":"2023-06-16","cited_by_count":337,"area":null},{"id":"2102.06810","title":"Understanding self-supervised Learning Dynamics without Contrastive Pairs","created":"2021-02-12","cited_by_count":256,"area":null},{"id":"2210.11466","title":"Surgical Fine-Tuning Improves Adaptation to Distribution Shifts","created":"2022-10-20","cited_by_count":205,"area":null},{"id":"2208.06677","title":"Adan: Adaptive Nesterov Momentum Algorithm for Faster Optimizing Deep Models","created":"2022-08-13","cited_by_count":174,"area":null},{"id":"2111.00034","title":"Neural Networks as Kernel Learners: The Silent Alignment Effect","created":"2021-10-29","cited_by_count":106,"area":null},{"id":"2208.12591","title":"On the Implicit Bias in Deep-Learning Algorithms","created":"2022-08-26","cited_by_count":86,"area":null},{"id":"2210.14199","title":"Same Pre-training Loss, Better Downstream: Implicit Bias Matters for Language Models","created":"2022-10-25","cited_by_count":84,"area":null},{"id":"2110.13905","title":"Gradient Descent on Two-layer Nets: Margin Maximization and Simplicity Bias","created":"2021-10-26","cited_by_count":83,"area":null},{"id":"2102.09769","title":"On the Implicit Bias of Initialization Shape: Beyond Infinitesimal Mirror Descent","created":"2021-02-19","cited_by_count":70,"area":null},{"id":"2409.17858","title":"How Feature Learning Can Improve Neural Scaling Laws","created":"2024-09-26","cited_by_count":56,"area":null},{"id":"2201.12760","title":"Implicit Regularization Towards Rank Minimization in ReLU Networks","created":"2022-01-30","cited_by_count":53,"area":null},{"id":"2306.07042","title":"Transformers learn through gradual rank increase","created":"2023-06-12","cited_by_count":52,"area":null},{"id":"2105.02716","title":"Noether's Learning Dynamics: Role of Symmetry Breaking in Neural Networks","created":"2021-05-06","cited_by_count":50,"area":null},{"id":"2110.03677","title":"Large Learning Rate Tames Homogeneity: Convergence and Balancing Effect","created":"2021-10-07","cited_by_count":50,"area":null},{"id":"2211.08422","title":"Mechanistic Mode Connectivity","created":"2022-11-15","cited_by_count":50,"area":null},{"id":"2501.16265","title":"Training Dynamics of In-Context Learning in Linear Attention","created":"2025-01-27","cited_by_count":46,"area":null},{"id":"2106.14289","title":"Global Convergence of Gradient Descent for Asymmetric Low-Rank Matrix Factorization","created":"2021-06-27","cited_by_count":45,"area":null},{"id":"2307.00144","title":"Abide by the Law and Follow the Flow: Conservation Laws for Gradient Flows","created":"2023-06-30","cited_by_count":45,"area":null},{"id":"2102.09972","title":"Implicit Regularization in Tensor Factorization","created":"2021-02-19","cited_by_count":44,"area":null},{"id":"2406.06158","title":"Get rich quick: exact solutions reveal how unbalanced initializations promote rapid feature learning","created":"2024-06-10","cited_by_count":42,"area":null},{"id":"2103.07601","title":"Approximating How Single Head Attention Learns","created":"2021-03-13","cited_by_count":38,"area":null},{"id":"2209.15055","title":"Implicit Bias of Large Depth Networks: a Notion of Rank for Nonlinear Functions","created":"2022-09-29","cited_by_count":38,"area":null},{"id":"2303.15438","title":"On the Stepwise Nature of Self-Supervised Learning","created":"2023-03-27","cited_by_count":36,"area":null},{"id":"2506.13018","title":"Symmetry in Neural Network Parameter Spaces","created":"2025-06-16","cited_by_count":36,"area":null},{"id":"2201.12680","title":"Understanding Deep Contrastive Learning via Coordinate-wise Optimization","created":"2022-01-29","cited_by_count":35,"area":null},{"id":"2210.17216","title":"Symmetries, flat minima, and the conserved quantities of gradient flow","created":"2022-10-31","cited_by_count":35,"area":null},{"id":"2107.06608","title":"Continuous vs. Discrete Optimization of Deep Neural Networks","created":"2021-07-14","cited_by_count":34,"area":null},{"id":"2110.02732","title":"On Margin Maximization in Linear and ReLU Networks","created":"2021-10-06","cited_by_count":34,"area":null},{"id":"2203.02839","title":"Algorithmic Regularization in Model-free Overparametrized Asymmetric Matrix Factorization","created":"2022-03-06","cited_by_count":34,"area":null},{"id":"2201.11968","title":"Training invariances and the low-rank phenomenon: beyond linear networks","created":"2022-01-28","cited_by_count":33,"area":null},{"id":"2408.11804","title":"Approaching Deep Learning through the Spectral Dynamics of Weights","created":"2024-08-21","cited_by_count":31,"area":null},{"id":"2604.21691","title":"There Will Be a Scientific Theory of Deep Learning","created":"2026-04-23","cited_by_count":29,"area":null},{"id":"2203.03756","title":"Flat minima generalize for low-rank matrix recovery","created":"2022-03-07","cited_by_count":28,"area":null},{"id":"2307.12851","title":"Early Neuron Alignment in Two-layer ReLU Networks with Small Initialization","created":"2023-07-24","cited_by_count":28,"area":null},{"id":"2201.11729","title":"Implicit Regularization in Hierarchical Tensor Factorization and Deep Convolutional Neural Networks","created":"2022-01-27","cited_by_count":27,"area":null},{"id":"2105.14301","title":"A Theory of Neural Tangent Kernel Alignment and Its Influence on Training","created":"2021-05-29","cited_by_count":25,"area":null},{"id":"2205.10637","title":"Symmetry Teleportation for Accelerated Optimization","created":"2022-05-21","cited_by_count":25,"area":null},{"id":"2302.10034","title":"Over-Parameterization Exponentially Slows Down Gradient Descent for Learning a Single Neuron","created":"2023-02-20","cited_by_count":25,"area":null},{"id":"2310.17087","title":"Good regularity creates large learning rate implicit biases: edge of stability, balancing, and catapult","created":"2023-10-26","cited_by_count":25,"area":null},{"id":"2406.10685","title":"Scale Equivariant Graph Metanetworks","created":"2024-06-15","cited_by_count":24,"area":null},{"id":"2305.13404","title":"Improving Convergence and Generalization Using Parameter Symmetries","created":"2023-05-22","cited_by_count":23,"area":null},{"id":"2108.02040","title":"Convergence of gradient descent for learning linear neural networks","created":"2021-08-04","cited_by_count":22,"area":null},{"id":"2210.11075","title":"Freeze then Train: Towards Provable Representation Learning under Spurious Correlations and Feature Noise","created":"2022-10-20","cited_by_count":22,"area":null},{"id":"2306.02601","title":"Aiming towards the minimizers: fast convergence of SGD for overparametrized problems","created":"2023-06-05","cited_by_count":22,"area":null},{"id":"2305.13064","title":"Gradient Descent Monotonically Decreases the Sharpness of Gradient Flow Solutions in Scalar Networks and Beyond","created":"2023-05-22","cited_by_count":21,"area":null},{"id":"2501.18915","title":"Algebra Unveils Deep Learning -- An Invitation to Neuroalgebraic Geometry","created":"2025-01-31","cited_by_count":21,"area":null},{"id":"2211.16980","title":"Infinite-width limit of deep linear neural networks","created":"2022-11-29","cited_by_count":20,"area":null},{"id":"2506.03133","title":"PoLAR: Polar-Decomposed Low-Rank Adapter Representation","created":"2025-06-03","cited_by_count":20,"area":null},{"id":"2102.02805","title":"How do Quadratic Regularizers Prevent Catastrophic Forgetting: The Role of Interpolation","created":"2021-02-04","cited_by_count":19,"area":null},{"id":"2104.05508","title":"Noether: The More Things Change, the More Stay the Same","created":"2021-04-12","cited_by_count":19,"area":null},{"id":"2206.01342","title":"Understanding the Role of Nonlinearity in Training Dynamics of Contrastive Learning","created":"2022-06-02","cited_by_count":19,"area":null},{"id":"2302.05779","title":"How to prepare your task head for finetuning","created":"2023-02-11","cited_by_count":19,"area":null},{"id":"2402.07193","title":"Parameter Symmetry and Noise Equilibrium of Stochastic Gradient Descent","created":"2024-02-11","cited_by_count":19,"area":null},{"id":"2209.07263","title":"Robustness in deep learning: The good (width), the bad (depth), and the ugly (initialization)","created":"2022-09-15","cited_by_count":18,"area":null},{"id":"2305.16427","title":"Neural (Tangent Kernel) Collapse","created":"2023-05-25","cited_by_count":18,"area":null},{"id":"2305.16534","title":"Variation Spaces for Multi-Output Neural Networks: Insights on Multi-Task Learning and Network Compression","created":"2023-05-25","cited_by_count":18,"area":null},{"id":"2310.01769","title":"How Over-Parameterization Slows Down Gradient Descent in Matrix Sensing: The Curses of Symmetry and Initialization","created":"2023-10-03","cited_by_count":18,"area":null},{"id":"2302.07384","title":"The Geometry of Neural Nets' Parameter Spaces Under Reparametrization","created":"2023-02-14","cited_by_count":16,"area":null},{"id":"2306.06479","title":"Learning a Neuron by a Shallow ReLU Network: Dynamics and Implicit Bias for Correlated Inputs","created":"2023-06-10","cited_by_count":16,"area":null},{"id":"2502.00264","title":"Beyond the Permutation Symmetry of Transformers: The Role of Rotation for Model Fusion","created":"2025-02-01","cited_by_count":16,"area":null},{"id":"2102.12430","title":"Noisy Gradient Descent Converges to Flat Minima for Nonconvex Matrix Factorization","created":"2021-02-24","cited_by_count":15,"area":null},{"id":"2210.06671","title":"Wasserstein Barycenter-based Model Fusion and Linear Mode Connectivity of Neural Networks","created":"2022-10-13","cited_by_count":15,"area":null},{"id":"2402.19262","title":"Masks, Signs, And Learning Rate Rewinding","created":"2024-02-29","cited_by_count":15,"area":null},{"id":"2410.18965","title":"On the Crucial Role of Initialization for Matrix Factorization","created":"2024-10-24","cited_by_count":15,"area":null},{"id":"2506.06489","title":"Alternating Gradient Flows: A Theory of Feature Learning in Two-layer Neural Networks","created":"2025-06-06","cited_by_count":14,"area":null},{"id":"2311.04163","title":"Outliers with Opposing Signals Have an Outsized Effect on Neural Network Optimization","created":"2023-11-07","cited_by_count":13,"area":null},{"id":"2301.00008","title":"Effects of Data Geometry in Early Deep Learning","created":"2022-12-29","cited_by_count":12,"area":null},{"id":"2305.06927","title":"Convergence of Alternating Gradient Descent for Matrix Factorization","created":"2023-05-11","cited_by_count":12,"area":null},{"id":"2502.05300","title":"Parameter Symmetry Potentially Unifies Deep Learning Theory","created":"2025-02-07","cited_by_count":12,"area":null},{"id":"2107.04479","title":"Convergence analysis for gradient flows in the training of artificial neural networks with ReLU activation","created":"2021-07-09","cited_by_count":11,"area":null},{"id":"2302.01952","title":"On a continuous time model of gradient descent dynamics and instability in deep learning","created":"2023-02-03","cited_by_count":11,"area":null},{"id":"2405.12888","title":"Keep the Momentum: Conservation Laws beyond Euclidean Gradient Flows","created":"2024-05-21","cited_by_count":11,"area":null},{"id":"2405.17299","title":"Simplicity Bias of Two-Layer Networks beyond Linearly Separable Data","created":"2024-05-27","cited_by_count":11,"area":null},{"id":"2105.05553","title":"Principal Components Bias in Over-parameterized Linear Models, and its Manifestation in Deep Neural Networks","created":"2021-05-12","cited_by_count":10,"area":null},{"id":"2310.07235","title":"Are GATs Out of Balance?","created":"2023-10-11","cited_by_count":10,"area":null},{"id":"2410.14802","title":"Implicit Regularization of Sharpness-Aware Minimization for Scale-Invariant Problems","created":"2024-10-18","cited_by_count":10,"area":null},{"id":"2303.15256","title":"Active Self-Supervised Learning: A Few Low-Cost Relationships Are All You Need","created":"2023-03-27","cited_by_count":9,"area":null},{"id":"2305.11615","title":"SFP: Spurious Feature-targeted Pruning for Out-of-Distribution Generalization","created":"2023-05-19","cited_by_count":9,"area":null},{"id":"2403.05293","title":"Leveraging Continuous Time to Understand Momentum When Training Diagonal Linear Networks","created":"2024-03-08","cited_by_count":9,"area":null},{"id":"2505.12387","title":"Neural Thermodynamics: Entropic Forces in Deep and Universal Representation Learning","created":"2025-05-18","cited_by_count":9,"area":null},{"id":"2505.18877","title":"RefLoRA: Refactored Low-Rank Adaptation for Efficient Fine-Tuning of Large Models","created":"2025-05-24","cited_by_count":9,"area":null},{"id":"2209.10675","title":"A Validation Approach to Over-parameterized Matrix and Image Recovery","created":"2022-09-21","cited_by_count":8,"area":null},{"id":"2301.12540","title":"Implicit Regularization for Group Sparsity","created":"2023-01-29","cited_by_count":8,"area":null},{"id":"2410.11474","title":"How Transformers Get Rich: Approximation and Dynamics Analysis","created":"2024-10-15","cited_by_count":8,"area":null},{"id":"2507.01098","title":"Proof of a perfect platonic representation hypothesis","created":"2025-07-01","cited_by_count":8,"area":null},{"id":"2303.03027","title":"Critical Points and Convergence Analysis of Generative Deep Linear Networks Trained with Bures-Wasserstein Loss","created":"2023-03-06","cited_by_count":7,"area":null},{"id":"2506.06194","title":"Transformative or Conservative? Conservation laws for ResNets and Transformers","created":"2025-06-06","cited_by_count":7,"area":null},{"id":"2509.25351","title":"Gradient Descent with Large Step Sizes: Chaos and Fractal Convergence Region","created":"2025-09-29","cited_by_count":7,"area":null},{"id":"2310.06159","title":"Provably Accelerating Ill-Conditioned Low-rank Estimation via Scaled Gradient Descent, Even with Overparameterization","created":"2023-10-09","cited_by_count":6,"area":null},{"id":"2410.14837","title":"Topological obstruction to the training of shallow ReLU neural networks","created":"2024-10-18","cited_by_count":6,"area":null},{"id":"2602.00693","title":"Topology and Geometry of the Learning Space of ReLU Networks: Connectivity and Singularities","created":"2026-01-31","cited_by_count":6,"area":null},{"id":"2602.19799","title":"Path-conditioned training: a principled way to rescale ReLU neural networks","created":"2026-02-23","cited_by_count":6,"area":null},{"id":"2104.10790","title":"Sharp Global Guarantees for Nonconvex Low-rank Recovery in the Noisy Overparameterized Regime","created":"2021-04-21","cited_by_count":5,"area":null},{"id":"2106.15456","title":"A Mechanism for Producing Aligned Latent Spaces with Autoencoders","created":"2021-06-29","cited_by_count":5,"area":null},{"id":"2111.08706","title":"How and When Random Feedback Works: A Case Study of Low-Rank Matrix Factorization","created":"2021-11-17","cited_by_count":5,"area":null},{"id":"2209.13394","title":"Magnitude and Angle Dynamics in Training Single ReLU Neurons","created":"2022-09-27","cited_by_count":5,"area":null},{"id":"2405.15942","title":"Can Implicit Bias Imply Adversarial Robustness?","created":"2024-05-24","cited_by_count":5,"area":null},{"id":"2502.17340","title":"Low-rank bias, weight decay, and model merging in neural networks","created":"2025-02-24","cited_by_count":5,"area":null},{"id":"2108.01772","title":"Nonconvex Factorization and Manifold Formulations are Almost Equivalent in Low-rank Matrix Optimization","created":"2021-08-03","cited_by_count":4,"area":null}],"next_cursor":"100"}