{"total":342,"rows":[{"id":"2202.10054","title":"Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution","created":"2022-02-21","cited_by_count":739,"area":null},{"id":"2105.02375","title":"A Geometric Analysis of Neural Collapse with Unconstrained Features","created":"2021-05-06","cited_by_count":227,"area":null},{"id":"2006.13198","title":"Spectral Bias and Task-Model Alignment Explain Generalization in Kernel Regression and Infinitely Wide Neural Networks","created":"2020-06-23","cited_by_count":166,"area":"alignment"},{"id":"2103.10427","title":"The Low-Rank Simplicity Bias in Deep Networks","created":"2021-03-18","cited_by_count":154,"area":null},{"id":"2206.06232","title":"Towards Understanding Sharpness-Aware Minimization","created":"2022-06-13","cited_by_count":146,"area":null},{"id":"2302.03025","title":"A Toy Model of Universality: Reverse Engineering How Networks Learn Group Operations","created":"2023-02-06","cited_by_count":144,"area":"mechinterp"},{"id":"2105.14368","title":"Fit without fear: remarkable mathematical phenomena of deep learning through the prism of interpolation","created":"2021-05-29","cited_by_count":132,"area":null},{"id":"2106.13799","title":"Assessing Generalization of SGD via Disagreement","created":"2021-06-25","cited_by_count":117,"area":null},{"id":"2207.14484","title":"Adaptive Gradient Methods at the Edge of Stability","created":"2022-07-29","cited_by_count":101,"area":null},{"id":"2202.14026","title":"Robust Training under Label Noise by Over-parameterization","created":"2022-02-28","cited_by_count":93,"area":null},{"id":"2105.05612","title":"Evading the Simplicity Bias: Training a Diverse Set of Models Discovers Solutions with Superior OOD Generalization","created":"2021-05-12","cited_by_count":83,"area":null},{"id":"2104.08938","title":"On the approximation of functions by tanh neural networks","created":"2021-04-18","cited_by_count":81,"area":null},{"id":"2402.10877","title":"Robust agents learn causal world models","created":"2024-02-16","cited_by_count":81,"area":"wm"},{"id":"2206.00939","title":"Gradient flow dynamics of shallow ReLU networks for square loss and orthogonal inputs","created":"2022-06-02","cited_by_count":76,"area":null},{"id":"2112.01388","title":"Residual Pathway Priors for Soft Equivariance Constraints","created":"2021-12-02","cited_by_count":75,"area":null},{"id":"2105.14602","title":"On the geometry of generalization and memorization in deep neural networks","created":"2021-05-30","cited_by_count":73,"area":null},{"id":"2201.03916","title":"Automated Reinforcement Learning (AutoRL): A Survey and Open Problems","created":"2022-01-11","cited_by_count":70,"area":null},{"id":"2204.03632","title":"The Effects of Regularization and Data Augmentation are Class Dependent","created":"2022-04-07","cited_by_count":67,"area":null},{"id":"2103.05127","title":"Model Complexity of Deep Learning: A Survey","created":"2021-03-08","cited_by_count":65,"area":null},{"id":"2310.16228","title":"On the Foundations of Shortcut Learning","created":"2023-10-24","cited_by_count":65,"area":null},{"id":"2207.02628","title":"The alignment property of SGD noise and how it helps select flat minima: A stability analysis","created":"2022-07-06","cited_by_count":64,"area":null},{"id":"2209.14863","title":"Neural Networks Efficiently Learn Low-Dimensional Representations with SGD","created":"2022-09-29","cited_by_count":63,"area":null},{"id":"2103.03014","title":"Lost in Pruning: The Effects of Pruning Neural Networks beyond Test Accuracy","created":"2021-03-04","cited_by_count":62,"area":null},{"id":"2105.03361","title":"What Kinds of Functions do Deep Neural Networks Learn? Insights from Variational Spline Theory","created":"2021-05-07","cited_by_count":60,"area":null},{"id":"2109.02355","title":"A Farewell to the Bias-Variance Tradeoff? An Overview of the Theory of Overparameterized Machine Learning","created":"2021-09-06","cited_by_count":57,"area":null},{"id":"2106.09276","title":"Uniform Convergence of Interpolators: Gaussian Width, Norm Bounds, and Benign Overfitting","created":"2021-06-17","cited_by_count":54,"area":null},{"id":"2309.08534","title":"Towards Last-layer Retraining for Group Robustness with Fewer Annotations","created":"2023-09-15","cited_by_count":54,"area":null},{"id":"2201.12760","title":"Implicit Regularization Towards Rank Minimization in ReLU Networks","created":"2022-01-30","cited_by_count":53,"area":null},{"id":"2110.03677","title":"Large Learning Rate Tames Homogeneity: Convergence and Balancing Effect","created":"2021-10-07","cited_by_count":50,"area":null},{"id":"2211.11567","title":"Neural networks trained with SGD learn distributions of increasing complexity","created":"2022-11-21","cited_by_count":50,"area":null},{"id":"2203.12677","title":"Vision-Based Manipulators Need to Also See from Their Hands","created":"2022-03-15","cited_by_count":47,"area":null},{"id":"2307.00144","title":"Abide by the Law and Follow the Flow: Conservation Laws for Gradient Flows","created":"2023-06-30","cited_by_count":45,"area":null},{"id":"2305.17490","title":"The Implicit Regularization of Dynamical Stability in Stochastic Gradient Descent","created":"2023-05-27","cited_by_count":44,"area":null},{"id":"2305.18761","title":"Identifying Spurious Biases Early in Training through the Lens of Simplicity Bias","created":"2023-05-30","cited_by_count":44,"area":null},{"id":"2403.02241","title":"Neural Redshift: Random Networks are not Random Functions","created":"2024-03-04","cited_by_count":43,"area":null},{"id":"2102.01117","title":"SGD Generalizes Better Than GD (And Regularization Doesn't Help)","created":"2021-02-01","cited_by_count":38,"area":null},{"id":"2110.02424","title":"Spectral Bias in Practice: The Role of Function Frequency in Generalization","created":"2021-10-06","cited_by_count":37,"area":null},{"id":"2301.12832","title":"Deep networks for system identification: a Survey","created":"2023-01-30","cited_by_count":37,"area":null},{"id":"2306.13575","title":"Scaling MLPs: A Tale of Inductive Bias","created":"2023-06-23","cited_by_count":37,"area":null},{"id":"2207.06569","title":"Benign, Tempered, or Catastrophic: A Taxonomy of Overfitting","created":"2022-07-14","cited_by_count":36,"area":null},{"id":"2302.08982","title":"(S)GD over Diagonal Linear Networks: Implicit Regularisation, Large Stepsizes and Edge of Stability","created":"2023-02-17","cited_by_count":36,"area":null},{"id":"2305.18764","title":"When Does Optimizing a Proper Loss Yield Calibration?","created":"2023-05-30","cited_by_count":36,"area":null},{"id":"2406.10650","title":"The Implicit Bias of Adam on Separable Data","created":"2024-06-15","cited_by_count":36,"area":null},{"id":"2106.05932","title":"Early-stopped neural networks are consistent","created":"2021-06-10","cited_by_count":34,"area":null},{"id":"2110.02732","title":"On Margin Maximization in Linear and ReLU Networks","created":"2021-10-06","cited_by_count":34,"area":null},{"id":"2107.00595","title":"Fast Margin Maximization via Dual Acceleration","created":"2021-07-01","cited_by_count":33,"area":null},{"id":"2310.18988","title":"A U-turn on Double Descent: Rethinking Parameter Counting in Statistical Learning","created":"2023-10-29","cited_by_count":33,"area":null},{"id":"2106.07880","title":"Scaling Neural Tangent Kernels via Sketching and Random Features","created":"2021-06-15","cited_by_count":32,"area":null},{"id":"2108.04552","title":"The Benefits of Implicit Regularization from SGD in Least Squares Problems","created":"2021-08-10","cited_by_count":32,"area":null},{"id":"2210.13738","title":"Pruning's Effect on Generalization Through the Lens of Training and Regularization","created":"2022-10-25","cited_by_count":31,"area":null},{"id":"2408.11804","title":"Approaching Deep Learning through the Spectral Dynamics of Weights","created":"2024-08-21","cited_by_count":31,"area":null},{"id":"2102.03349","title":"On the Reproducibility of Neural Network Predictions","created":"2021-02-05","cited_by_count":29,"area":null},{"id":"2109.03810","title":"Scaled ReLU Matters for Training Vision Transformers","created":"2021-09-08","cited_by_count":29,"area":null},{"id":"2202.13441","title":"Stability vs Implicit Bias of Gradient Methods on Separable Data and Beyond","created":"2022-02-27","cited_by_count":29,"area":null},{"id":"2211.05641","title":"Regression as Classification: Influence of Task Formulation on Neural Network Features","created":"2022-11-10","cited_by_count":29,"area":null},{"id":"2108.04190","title":"On the Power of Differentiable Learning versus PAC and SQ Learning","created":"2021-08-09","cited_by_count":28,"area":null},{"id":"2109.14509","title":"PAC-Bayes Information Bottleneck","created":"2021-09-29","cited_by_count":28,"area":null},{"id":"2203.03756","title":"Flat minima generalize for low-rank matrix recovery","created":"2022-03-07","cited_by_count":28,"area":null},{"id":"2205.07069","title":"Homogenization of SGD in high-dimensions: Exact dynamics and generalization properties","created":"2022-05-14","cited_by_count":27,"area":null},{"id":"2309.03160","title":"ResFields: Residual Neural Fields for Spatiotemporal Signals","created":"2023-09-06","cited_by_count":27,"area":null},{"id":"2205.12808","title":"Mirror Descent Maximizes Generalized Margin and Can Be Implemented Efficiently","created":"2022-05-25","cited_by_count":26,"area":null},{"id":"2206.08684","title":"Sparse Double Descent: Where Network Pruning Aggravates Overfitting","created":"2022-06-17","cited_by_count":26,"area":null},{"id":"2406.04112","title":"Compressible Dynamics in Deep Overparameterized Low-Rank Learning & Adaptation","created":"2024-06-06","cited_by_count":26,"area":null},{"id":"2110.05518","title":"Global Optimality Beyond Two Layers: Training Deep ReLU Networks via Convex Programs","created":"2021-10-11","cited_by_count":25,"area":null},{"id":"2203.10036","title":"On the Generalization Mystery in Deep Learning","created":"2022-03-18","cited_by_count":25,"area":null},{"id":"2302.00849","title":"Implicit regularization in Heavy-ball momentum accelerated stochastic gradient descent","created":"2023-02-02","cited_by_count":25,"area":null},{"id":"2309.16932","title":"Symmetry Induces Structure and Constraint of Learning","created":"2023-09-29","cited_by_count":25,"area":null},{"id":"2112.03215","title":"Multi-scale Feature Learning Dynamics: Insights for Double Descent","created":"2021-12-06","cited_by_count":24,"area":null},{"id":"2112.11027","title":"More is Less: Inducing Sparsity via Overparameterization","created":"2021-12-21","cited_by_count":24,"area":null},{"id":"2306.08321","title":"Nonparametric regression using over-parameterized shallow ReLU neural networks","created":"2023-06-14","cited_by_count":24,"area":null},{"id":"2206.07252","title":"Implicit Regularization or Implicit Conditioning? Exact Risk Trajectories of SGD in High Dimensions","created":"2022-06-15","cited_by_count":23,"area":null},{"id":"2110.03891","title":"Does Momentum Change the Implicit Regularization on Separable Data?","created":"2021-10-08","cited_by_count":22,"area":null},{"id":"2212.04590","title":"Learning Options via Compression","created":"2022-12-08","cited_by_count":22,"area":null},{"id":"2103.01499","title":"Demystifying Batch Normalization in ReLU Networks: Equivalent Convex Optimization Models and Implicit Regularization","created":"2021-03-02","cited_by_count":21,"area":null},{"id":"2205.09072","title":"On the Effective Number of Linear Regions in Shallow Univariate ReLU Networks: Convergence Guarantees and Implicit Bias","created":"2022-05-18","cited_by_count":21,"area":null},{"id":"2301.09554","title":"Deep Learning Meets Sparse Regularization: A Signal Processing Perspective","created":"2023-01-23","cited_by_count":21,"area":null},{"id":"2308.13431","title":"Six Lectures on Linearized Neural Networks","created":"2023-08-25","cited_by_count":20,"area":null},{"id":"2309.01213","title":"Implicit regularization of deep residual networks towards neural ODEs","created":"2023-09-03","cited_by_count":20,"area":null},{"id":"2401.07187","title":"A Survey on Statistical Theory of Deep Learning: Approximation, Training Dynamics, and Generative Models","created":"2024-01-14","cited_by_count":20,"area":null},{"id":"2403.11391","title":"Investigating the Benefits of Projection Head for Representation Learning","created":"2024-03-18","cited_by_count":20,"area":null},{"id":"2202.13361","title":"Benign Underfitting of Stochastic Gradient Descent","created":"2022-02-27","cited_by_count":19,"area":null},{"id":"2303.01353","title":"Penalising the biases in norm regularisation enforces sparsity","created":"2023-03-02","cited_by_count":19,"area":null},{"id":"2102.12238","title":"Inductive Bias of Multi-Channel Linear Convolutional Networks with Bounded Weight Norm","created":"2021-02-24","cited_by_count":18,"area":null},{"id":"2108.11489","title":"The Interplay Between Implicit Bias and Benign Overfitting in Two-Layer Linear Networks","created":"2021-08-25","cited_by_count":18,"area":null},{"id":"2203.01572","title":"Data Augmentation as Feature Manipulation","created":"2022-03-03","cited_by_count":18,"area":null},{"id":"2205.07019","title":"Evaluating the Generalization Ability of Super-Resolution Networks","created":"2022-05-14","cited_by_count":18,"area":null},{"id":"2305.16534","title":"Variation Spaces for Multi-Output Neural Networks: Insights on Multi-Task Learning and Network Compression","created":"2023-05-25","cited_by_count":18,"area":null},{"id":"2311.14646","title":"More is Better in Modern Machine Learning: when Infinite Overparameterization is Optimal and Overfitting is Obligatory","created":"2023-11-24","cited_by_count":18,"area":null},{"id":"2410.22069","title":"Flavors of Margin: Implicit Bias of Steepest Descent in Homogeneous Neural Networks","created":"2024-10-29","cited_by_count":18,"area":null},{"id":"2112.04470","title":"Optimistic Rates: A Unifying Theory for Interpolation Learning and Regularization in Linear Regression","created":"2021-12-08","cited_by_count":17,"area":null},{"id":"2202.06915","title":"Stochastic linear optimization never overfits with quadratically-bounded losses on general data","created":"2022-02-14","cited_by_count":17,"area":null},{"id":"2203.06498","title":"The worst of both worlds: A comparative analysis of errors in learning from data in psychology and machine learning","created":"2022-03-12","cited_by_count":17,"area":null},{"id":"2205.13647","title":"Learning to Reason with Neural Networks: Generalization, Unseen Data and Boolean Measures","created":"2022-05-26","cited_by_count":17,"area":null},{"id":"2302.04925","title":"Information Theoretic Lower Bounds for Information Theoretic Upper Bounds","created":"2023-02-09","cited_by_count":17,"area":null},{"id":"2310.06982","title":"Data Distillation Can Be Like Vodka: Distilling More Times For Better Quality","created":"2023-10-10","cited_by_count":17,"area":null},{"id":"2303.15809","title":"Kernel interpolation generalizes poorly","created":"2023-03-28","cited_by_count":16,"area":null},{"id":"2409.16434","title":"Lessons and Insights from a Unifying Study of Parameter-Efficient Fine-Tuning (PEFT) in Visual Recognition","created":"2024-09-24","cited_by_count":16,"area":null},{"id":"2102.02976","title":"Generalization Bounds for Noisy Iterative Algorithms Using Properties of Additive Noise Channels","created":"2021-02-05","cited_by_count":15,"area":null},{"id":"2109.07421","title":"Gaussian Moments as Physically Inspired Molecular Descriptors for Accurate and Scalable Machine Learning Potentials","created":"2021-09-15","cited_by_count":15,"area":null},{"id":"2102.05242","title":"Patterns, predictions, and actions: A story about machine learning","created":"2021-02-10","cited_by_count":14,"area":null}],"next_cursor":"100"}