{"list":"other","total":71,"rows":[{"id":"1810.04805","title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding","created":"2018-10-11","cited_by_count":40399},{"id":"1412.6980","title":"Adam: A Method for Stochastic Optimization","created":"2014-12-22","cited_by_count":37240},{"id":"1806.07572","title":"Neural Tangent Kernel: Convergence and Generalization in Neural Networks","created":"2018-06-20","cited_by_count":2621},{"id":"1812.07956","title":"On Lazy Training in Differentiable Programming","created":"2018-12-19","cited_by_count":675},{"id":"1810.02054","title":"Gradient Descent Provably Optimizes Over-parameterized Neural Networks","created":"2018-10-04","cited_by_count":659},{"id":"1711.00165","title":"Deep Neural Networks as Gaussian Processes","created":"2017-11-01","cited_by_count":622},{"id":"1811.03804","title":"Gradient Descent Finds Global Minima of Deep Neural Networks","created":"2018-11-09","cited_by_count":574},{"id":"1904.11955","title":"On Exact Computation with an Infinitely Wide Neural Net","created":"2019-04-26","cited_by_count":573},{"id":"1804.06561","title":"A Mean Field View of the Landscape of Two-Layers Neural Networks","created":"2018-04-18","cited_by_count":466},{"id":"1805.09545","title":"On the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal Transport","created":"2018-05-24","cited_by_count":426},{"id":"1908.05355","title":"The generalization error of random features regression: Precise asymptotics and double descent curve","created":"2019-08-14","cited_by_count":413},{"id":"1811.04918","title":"Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layers","created":"2018-11-12","cited_by_count":340},{"id":"2103.09177","title":"Deep learning: a statistical viewpoint","created":"2021-03-16","cited_by_count":240},{"id":"2003.00307","title":"Loss landscapes and optimization in over-parameterized non-linear systems and neural networks","created":"2020-02-29","cited_by_count":233},{"id":"2003.02218","title":"The large learning rate phase of deep learning: the catapult mechanism","created":"2020-03-04","cited_by_count":216},{"id":"1810.01075","title":"Implicit Self-Regularization in Deep Neural Networks: Evidence from Random Matrix Theory and Implications for Learning","created":"2018-10-02","cited_by_count":215},{"id":"2011.14522","title":"Feature Learning in Infinite-Width Neural Networks","created":"2020-11-30","cited_by_count":190},{"id":"1802.01396","title":"To understand deep learning we need to understand kernel learning","created":"2018-02-05","cited_by_count":187},{"id":"2010.15110","title":"Deep learning versus kernel learning: an empirical study of loss landscape geometry and the time evolution of the Neural Tangent Kernel","created":"2020-10-28","cited_by_count":187},{"id":"1905.12173","title":"On the Inductive Bias of Neural Tangent Kernels","created":"2019-05-29","cited_by_count":177},{"id":"1902.04674","title":"Towards moderate overparameterization: global convergence guarantees for training shallow neural networks","created":"2019-02-12","cited_by_count":176},{"id":"1907.04595","title":"Towards Explaining the Regularization Effect of Initial Large Learning Rate in Training Neural Networks","created":"2019-07-10","cited_by_count":162},{"id":"2206.15144","title":"Neural Networks can Learn Representations with Gradient Descent","created":"2022-06-30","cited_by_count":161},{"id":"2007.15801","title":"Finite Versus Infinite Neural Networks: an Empirical Study","created":"2020-07-31","cited_by_count":157},{"id":"1904.12191","title":"Linearized two-layers neural networks in high dimension","created":"2019-04-27","cited_by_count":155},{"id":"1910.12478","title":"Tensor Programs I: Wide Feedforward or Recurrent Neural Networks of Any Architecture are Gaussian Processes","created":"2019-10-28","cited_by_count":151},{"id":"2002.09572","title":"The Break-Even Point on Optimization Trajectories of Deep Neural Networks","created":"2020-02-21","cited_by_count":150},{"id":"2006.13409","title":"When Do Neural Networks Outperform Kernel Methods?","created":"2020-06-24","cited_by_count":150},{"id":"2205.01445","title":"High-dimensional Asymptotics of Feature Learning: How One Gradient Step Improves the Representation","created":"2022-05-03","cited_by_count":150},{"id":"1811.08888","title":"Stochastic Gradient Descent Optimizes Over-parameterized Deep ReLU Networks","created":"2018-11-21","cited_by_count":141},{"id":"2002.09339","title":"Generalisation error in learning with random features and the hidden manifold model","created":"2020-02-21","cited_by_count":127},{"id":"1909.12292","title":"Polylogarithmic width suffices for gradient descent to achieve arbitrarily small test error with shallow ReLU networks","created":"2019-09-26","cited_by_count":121},{"id":"2002.06716","title":"Predicting trends in the quality of state-of-the-art neural networks without access to training or testing data","created":"2020-02-17","cited_by_count":115},{"id":"2111.00034","title":"Neural Networks as Kernel Learners: The Silent Alignment Effect","created":"2021-10-29","cited_by_count":106},{"id":"1901.08276","title":"Traditional and Heavy-Tailed Self Regularization in Neural Network Models","created":"2019-01-24","cited_by_count":105},{"id":"2008.06786","title":"The Neural Tangent Kernel in High Dimensions: Triple Descent and a Multi-Scale Theory of Generalization","created":"2020-08-15","cited_by_count":101},{"id":"1909.08156","title":"Dynamics of Deep Neural Networks and Neural Tangent Hierarchy","created":"2019-09-18","cited_by_count":96},{"id":"2203.06176","title":"More Than a Toy: Random Matrix Models Predict How Real-World Neural Representations Generalize","created":"2022-03-11","cited_by_count":91},{"id":"1812.10004","title":"Overparameterized Nonlinear Learning: Gradient Descent Takes the Shortest Path?","created":"2018-12-25","cited_by_count":88},{"id":"1702.05419","title":"A Random Matrix Approach to Neural Networks","created":"2017-02-17","cited_by_count":80},{"id":"1910.01619","title":"Beyond Linearization: On Quadratic and Higher-Order Approximation of Wide Neural Networks","created":"2019-10-03","cited_by_count":79},{"id":"2002.07400","title":"Learning Parities with Neural Networks","created":"2020-02-18","cited_by_count":78},{"id":"1909.11304","title":"Asymptotics of Wide Networks from Feynman Diagrams","created":"2019-09-25","cited_by_count":77},{"id":"2005.11879","title":"Spectra of the Conjugate Kernel and Neural Tangent Kernel for linear-width neural networks","created":"2020-05-25","cited_by_count":77},{"id":"2012.04030","title":"Statistical Mechanics of Deep Linear Neural Networks: The Back-Propagating Kernel Renormalization","created":"2020-12-07","cited_by_count":71},{"id":"2006.05013","title":"A Random Matrix Analysis of Random Fourier Features: Beyond the Gaussian Kernel, a Precise Phase Transition, and the Corresponding Double Descent","created":"2020-06-09","cited_by_count":70},{"id":"2102.11742","title":"Classifying high-dimensional Gaussian mixtures: Where kernel methods fail and neural networks succeed","created":"2021-02-23","cited_by_count":69},{"id":"2210.05337","title":"SGD with Large Step Sizes Learns Sparse Features","created":"2022-10-11","cited_by_count":68},{"id":"1906.05392","title":"Generalization Guarantees for Neural Networks via Harnessing the Low-rank Structure of the Jacobian","created":"2019-06-12","cited_by_count":67},{"id":"2006.14599","title":"The Surprising Simplicity of the Early-Time Learning Dynamics of Neural Networks","created":"2020-06-25","cited_by_count":58},{"id":"1812.05994","title":"Products of Many Large Random Matrices and Gradients in Deep Neural Networks","created":"2018-12-14","cited_by_count":49},{"id":"2106.06770","title":"What can linearized neural networks actually say about generalization?","created":"2021-06-12","cited_by_count":49},{"id":"2101.09612","title":"On the Proof of Global Convergence of Gradient Descent for Deep ReLU Networks with Linear Widths","created":"2021-01-24","cited_by_count":44},{"id":"2002.10376","title":"The Two Regimes of Deep Network Training","created":"2020-02-24","cited_by_count":30},{"id":"1904.03090","title":"Eigenvalue distribution of nonlinear models of random matrices","created":"2019-04-05","cited_by_count":27},{"id":"2105.10585","title":"Properties of the After Kernel","created":"2021-05-21","cited_by_count":27},{"id":"2105.14301","title":"A Theory of Neural Tangent Kernel Alignment and Its Influence on Training","created":"2021-05-29","cited_by_count":25},{"id":"2203.00573","title":"Contrasting random and learned features in deep Bayesian linear regression","created":"2022-03-01","cited_by_count":24},{"id":"2203.16462","title":"Convergence of gradient descent for deep neural networks","created":"2022-03-30","cited_by_count":24},{"id":"2109.09304","title":"Deformed semicircle law and concentration of nonlinear random matrices for ultra-wide neural networks","created":"2021-09-20","cited_by_count":21},{"id":"2203.14661","title":"Random matrix analysis of deep neural network weight matrices","created":"2022-03-28","cited_by_count":21},{"id":"2202.02842","title":"Evaluating natural language processing models with generalization metrics that do not need access to any training or testing data","created":"2022-02-06","cited_by_count":19},{"id":"2010.11775","title":"Label-Aware Neural Tangent Kernel: Toward Better Generalization and Local Elasticity","created":"2020-10-22","cited_by_count":18},{"id":"2006.09717","title":"Neural Anisotropy Directions","created":"2020-06-17","cited_by_count":17},{"id":"2106.06251","title":"On Learnability via Gradient Method for Two-Layer ReLU Neural Networks in Teacher-Student Setting","created":"2021-06-11","cited_by_count":16},{"id":"2210.12030","title":"Evolution of Neural Tangent Kernels under Benign and Adversarial Training","created":"2022-10-21","cited_by_count":14},{"id":"2201.04753","title":"Largest Eigenvalues of the Conjugate Kernel of Single-Layered Neural Networks","created":"2022-01-13","cited_by_count":13},{"id":"2111.13331","title":"Impact of classification difficulty on the weight matrices spectra in Deep Learning and application to early-stopping","created":"2021-11-26","cited_by_count":12},{"id":"2005.07360","title":"Learning Rate Annealing Can Provably Help Generalization, Even for Convex Problems","created":"2020-05-15","cited_by_count":11},{"id":"2202.13733","title":"On the Benefits of Large Learning Rates for Kernel Methods","created":"2022-02-28","cited_by_count":8},{"id":"2201.12052","title":"Improved Overparametrization Bounds for Global Convergence of Stochastic Gradient Descent for Shallow Neural Networks","created":"2022-01-28","cited_by_count":1}],"next_cursor":null}