{"list":"other","total":92,"rows":[{"id":"1706.03762","title":"Attention Is All You Need","created":"2017-06-12","cited_by_count":60361},{"id":"2006.11239","title":"Denoising Diffusion Probabilistic Models","created":"2020-06-19","cited_by_count":21076},{"id":"1912.01703","title":"PyTorch: An Imperative Style, High-Performance Deep Learning Library","created":"2019-12-03","cited_by_count":20305},{"id":"1409.1556","title":"Very Deep Convolutional Networks for Large-Scale Image Recognition","created":"2014-09-04","cited_by_count":15364},{"id":"1503.03585","title":"Deep Unsupervised Learning using Nonequilibrium Thermodynamics","created":"2015-03-12","cited_by_count":7058},{"id":"1502.03167","title":"Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift","created":"2015-02-11","cited_by_count":7005},{"id":"2204.06125","title":"Hierarchical Text-Conditional Image Generation with CLIP Latents","created":"2022-04-13","cited_by_count":6585},{"id":"1312.6199","title":"Intriguing properties of neural networks","created":"2013-12-21","cited_by_count":5155},{"id":"2108.07258","title":"On the Opportunities and Risks of Foundation Models","created":"2021-08-16","cited_by_count":4033},{"id":"2307.15043","title":"Universal and Transferable Adversarial Attacks on Aligned Language Models","created":"2023-07-27","cited_by_count":3104},{"id":"1609.07843","title":"Pointer Sentinel Mixture Models","created":"2016-09-26","cited_by_count":3096},{"id":"1806.07572","title":"Neural Tangent Kernel: Convergence and Generalization in Neural Networks","created":"2018-06-20","cited_by_count":2621},{"id":"1803.03635","title":"The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks","created":"2018-03-09","cited_by_count":2262},{"id":"1711.05101","title":"Decoupled Weight Decay Regularization","created":"2017-11-14","cited_by_count":1831},{"id":"2304.01373","title":"Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling","created":"2023-04-03","cited_by_count":1786},{"id":"1907.07174","title":"Natural Adversarial Examples","created":"2019-07-16","cited_by_count":1408},{"id":"2203.05482","title":"Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time","created":"2022-03-10","cited_by_count":1396},{"id":"1806.08734","title":"On the Spectral Bias of Neural Networks","created":"2018-06-22","cited_by_count":1341},{"id":"2212.04089","title":"Editing Models with Task Arithmetic","created":"2022-12-08","cited_by_count":1120},{"id":"1905.02175","title":"Adversarial Examples Are Not Bugs, They Are Features","created":"2019-05-06","cited_by_count":953},{"id":"1812.11118","title":"Reconciling modern machine learning practice and the bias-variance trade-off","created":"2018-12-28","cited_by_count":927},{"id":"2301.05217","title":"Progress measures for grokking via mechanistic interpretability","created":"2023-01-12","cited_by_count":839},{"id":"1312.6120","title":"Exact solutions to the nonlinear dynamics of learning in deep linear neural networks","created":"2013-12-20","cited_by_count":763},{"id":"1912.02292","title":"Deep Double Descent: Where Bigger Models and More Data Hurt","created":"2019-12-04","cited_by_count":672},{"id":"1912.05671","title":"Linear Mode Connectivity and the Lottery Ticket Hypothesis","created":"2019-12-11","cited_by_count":637},{"id":"2201.02177","title":"Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets","created":"2022-01-06","cited_by_count":629},{"id":"2109.02846","title":"Datasets: A Community Library for Natural Language Processing","created":"2021-09-07","cited_by_count":530},{"id":"1912.02178","title":"Fantastic Generalization Measures and Where to Find Them","created":"2019-12-04","cited_by_count":513},{"id":"2209.04836","title":"Git Re-Basin: Merging Models modulo Permutation Symmetries","created":"2022-09-11","cited_by_count":471},{"id":"1905.13655","title":"Implicit Regularization in Deep Matrix Factorization","created":"2019-05-31","cited_by_count":405},{"id":"2312.02696","title":"Analyzing and Improving the Training Dynamics of Diffusion Models","created":"2023-12-05","cited_by_count":402},{"id":"2008.11687","title":"What is being transferred in transfer learning?","created":"2020-08-26","cited_by_count":368},{"id":"1412.6614","title":"In Search of the Real Inductive Bias: On the Role of Implicit Regularization in Deep Learning","created":"2014-12-20","cited_by_count":342},{"id":"1705.09280","title":"Implicit Regularization in Matrix Factorization","created":"2017-05-25","cited_by_count":322},{"id":"2110.06296","title":"The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networks","created":"2021-10-12","cited_by_count":291},{"id":"1802.06509","title":"On the Optimization of Deep Networks: Implicit Acceleration by Overparameterization","created":"2018-02-19","cited_by_count":284},{"id":"2205.10343","title":"Towards Understanding Grokking: An Effective Theory of Representation Learning","created":"2022-05-20","cited_by_count":230},{"id":"1902.04742","title":"Uniform convergence may be unable to explain generalization in deep learning","created":"2019-02-13","cited_by_count":224},{"id":"1810.01075","title":"Implicit Self-Regularization in Deep Neural Networks: Evidence from Random Matrix Theory and Implications for Learning","created":"2018-10-02","cited_by_count":215},{"id":"1901.09321","title":"Fixup Initialization: Residual Learning Without Normalization","created":"2019-01-27","cited_by_count":198},{"id":"2208.03306","title":"Branch-Train-Merge: Embarrassingly Parallel Training of Expert Language Models","created":"2022-08-05","cited_by_count":191},{"id":"2010.15110","title":"Deep learning versus kernel learning: an empirical study of loss landscape geometry and the time evolution of the Neural Tangent Kernel","created":"2020-10-28","cited_by_count":187},{"id":"2207.08799","title":"Hidden Progress in Deep Learning: SGD Learns Parities Near the Computational Limit","created":"2022-07-18","cited_by_count":187},{"id":"1810.02032","title":"Gradient descent aligns the layers of deep linear networks","created":"2018-10-04","cited_by_count":182},{"id":"1806.00900","title":"Algorithmic Regularization in Learning Deep Homogeneous Models: Layers are Automatically Balanced","created":"2018-06-04","cited_by_count":177},{"id":"2103.10427","title":"The Low-Rank Simplicity Bias in Deep Networks","created":"2021-03-18","cited_by_count":154},{"id":"2012.09839","title":"Towards Resolving the Implicit Bias of Gradient Descent for Matrix Factorization: Greedy Low-Rank Learning","created":"2020-12-17","cited_by_count":150},{"id":"2205.09739","title":"Diverse Weight Averaging for Out-of-Distribution Generalization","created":"2022-05-19","cited_by_count":148},{"id":"2105.12221","title":"Geometry of the Loss Landscape in Overparameterized Neural Networks: Symmetries and Invariances","created":"2021-05-25","cited_by_count":127},{"id":"1706.05350","title":"L2 Regularization versus Batch and Weight Normalization","created":"2017-06-16","cited_by_count":124},{"id":"2211.08403","title":"REPAIR: REnormalizing Permuted Activations for Interpolation Repair","created":"2022-11-15","cited_by_count":120},{"id":"2005.06398","title":"Implicit Regularization in Deep Learning May Not Be Explainable by Norms","created":"2020-05-13","cited_by_count":119},{"id":"1810.12281","title":"Three Mechanisms of Weight Decay Regularization","created":"2018-10-29","cited_by_count":113},{"id":"2310.06110","title":"Grokking as the Transition from Lazy to Rich Training Dynamics","created":"2023-10-09","cited_by_count":108},{"id":"2206.04817","title":"The Slingshot Mechanism: An Empirical Study of Adaptive Optimizers and the Grokking Phenomenon","created":"2022-06-10","cited_by_count":94},{"id":"1906.05478","title":"Robust and interpretable blind image denoising via bias-free convolutional neural networks","created":"2019-06-13","cited_by_count":90},{"id":"2301.02679","title":"Grokking modular arithmetic","created":"2023-01-06","cited_by_count":83},{"id":"2303.11873","title":"A Tale of Two Circuits: Grokking as Competition of Sparse and Dense Subnetworks","created":"2023-03-21","cited_by_count":83},{"id":"2311.18817","title":"Dichotomy of Early and Late Phase Implicit Biases Can Provably Induce Grokking","created":"2023-11-30","cited_by_count":80},{"id":"1802.09979","title":"The Emergence of Spectral Universality in Deep Networks","created":"2018-02-27","cited_by_count":74},{"id":"2310.04415","title":"Why Do We Need Weight Decay in Modern Deep Learning?","created":"2023-10-06","cited_by_count":72},{"id":"1906.06247","title":"Explaining Landscape Connectivity of Low-cost Solutions for Multilayer Nets","created":"2019-06-14","cited_by_count":63},{"id":"1907.02911","title":"Weight-space symmetry in deep networks gives rise to permutation saddles, connected by equal-loss valleys across the loss landscape","created":"2019-07-05","cited_by_count":63},{"id":"2103.03936","title":"Pufferfish: Communication-efficient Models At No Extra Cost","created":"2021-03-05","cited_by_count":61},{"id":"2201.12760","title":"Implicit Regularization Towards Rank Minimization in ReLU Networks","created":"2022-01-30","cited_by_count":53},{"id":"2306.07042","title":"Transformers learn through gradual rank increase","created":"2023-06-12","cited_by_count":52},{"id":"2402.15555","title":"Deep Networks Always Grok and Here is Why","created":"2024-02-23","cited_by_count":51},{"id":"2303.06173","title":"Unifying Grokking and Double Descent","created":"2023-03-10","cited_by_count":50},{"id":"2206.06072","title":"Rank Diminishing in Deep Neural Networks","created":"2022-06-13","cited_by_count":47},{"id":"2210.07082","title":"Implicit Bias in Leaky ReLU Networks Trained on High-Dimensional Data","created":"2022-10-13","cited_by_count":46},{"id":"2012.05156","title":"Implicit Regularization in ReLU Networks with the Square Loss","created":"2020-12-09","cited_by_count":45},{"id":"1901.08278","title":"Heavy-Tailed Universality Predicts Trends in Test Accuracies for Very Large Pre-Trained Deep Neural Networks","created":"2019-01-24","cited_by_count":44},{"id":"2205.13571","title":"Low-rank lottery tickets: finding efficient low-rank neural networks via matrix differential equations","created":"2022-05-26","cited_by_count":43},{"id":"2306.04251","title":"Stochastic Collapse: How Gradient Noise Attracts SGD Dynamics Towards Simpler Subnetworks","created":"2023-06-07","cited_by_count":39},{"id":"2210.03044","title":"Unmasking the Lottery Ticket Hypothesis: What's Encoded in a Winning Ticket's Mask?","created":"2022-10-06","cited_by_count":38},{"id":"2306.13253","title":"Predicting Grokking Long Before it Happens: A look into the loss landscape of models which grok","created":"2023-06-23","cited_by_count":35},{"id":"2201.11968","title":"Training invariances and the low-rank phenomenon: beyond linear networks","created":"2022-01-28","cited_by_count":33},{"id":"2310.02541","title":"Benign Overfitting and Grokking in ReLU Networks for XOR Cluster Data","created":"2023-10-04","cited_by_count":33},{"id":"2305.16038","title":"Implicit bias of SGD in $L_{2}$-regularized linear DNNs: One-way jumps from high to low rank","created":"2023-05-25","cited_by_count":29},{"id":"2212.12147","title":"The Onset of Variance-Limited Behavior for Networks in the Lazy and Rich Regimes","created":"2022-12-23","cited_by_count":25},{"id":"2310.19103","title":"Proving Linear Mode Connectivity of Neural Networks via Optimal Transport","created":"2023-10-29","cited_by_count":25},{"id":"2301.09554","title":"Deep Learning Meets Sparse Regularization: A Signal Processing Perspective","created":"2023-01-23","cited_by_count":21},{"id":"2306.01154","title":"The Law of Parsimony in Gradient Descent for Learning Deep Linear Networks","created":"2023-06-01","cited_by_count":20},{"id":"2202.00856","title":"The Role of Linear Layers in Nonlinear Interpolating Networks","created":"2022-02-02","cited_by_count":16},{"id":"2110.09548","title":"Path Regularization: A Convexity and Sparsity Inducing Regularization for Parallel ReLU Networks","created":"2021-10-18","cited_by_count":14},{"id":"2311.06597","title":"Understanding Grokking Through A Robustness Viewpoint","created":"2023-11-11","cited_by_count":12},{"id":"2402.03991","title":"Provable Emergence of Deep Neural Collapse and Low-Rank Bias in $L^2$-Regularized Nonlinear Networks","created":"2024-02-06","cited_by_count":11},{"id":"1812.02566","title":"Singular Values for ReLU Layers","created":"2018-12-06","cited_by_count":10},{"id":"2402.04051","title":"Analysis of Linear Mode Connectivity via Permutation-Based Weight Matching: With Insights into Other Permutation Search Methods","created":"2024-02-06","cited_by_count":9},{"id":"2208.06894","title":"The SVD of Convolutional Weights: A CNN Interpretability Framework","created":"2022-08-14","cited_by_count":8},{"id":"2303.03374","title":"To Stay or Not to Stay in the Pre-train Basin: Insights on Ensembling in Transfer Learning","created":"2023-03-06","cited_by_count":6},{"id":"2105.01346","title":"Implicit Regularization in Deep Tensor Factorization","created":"2021-05-04","cited_by_count":5}],"next_cursor":null}