{"list":"other","total":56,"rows":[{"id":"1412.6980","title":"Adam: A Method for Stochastic Optimization","created":"2014-12-22","cited_by_count":37240},{"id":"1409.1556","title":"Very Deep Convolutional Networks for Large-Scale Image Recognition","created":"2014-09-04","cited_by_count":15364},{"id":"1607.06450","title":"Layer Normalization","created":"2016-07-21","cited_by_count":5669},{"id":"1609.04836","title":"On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima","created":"2016-09-15","cited_by_count":1352},{"id":"1712.09913","title":"Visualizing the Loss Landscape of Neural Nets","created":"2017-12-28","cited_by_count":1130},{"id":"1506.01186","title":"Cyclical Learning Rates for Training Neural Networks","created":"2015-06-03","cited_by_count":629},{"id":"2201.02177","title":"Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets","created":"2022-01-06","cited_by_count":629},{"id":"1811.03804","title":"Gradient Descent Finds Global Minima of Deep Neural Networks","created":"2018-11-09","cited_by_count":574},{"id":"1906.11300","title":"Benign Overfitting in Linear Regression","created":"2019-06-26","cited_by_count":549},{"id":"1602.07868","title":"Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks","created":"2016-02-25","cited_by_count":543},{"id":"1903.08560","title":"Surprises in High-Dimensional Ridgeless Least Squares Interpolation","created":"2019-03-19","cited_by_count":513},{"id":"2103.00065","title":"Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability","created":"2021-02-26","cited_by_count":427},{"id":"1711.00489","title":"Don't Decay the Learning Rate, Increase the Batch Size","created":"2017-11-01","cited_by_count":320},{"id":"1906.05890","title":"Gradient Descent Maximizes the Margin of Homogeneous Neural Networks","created":"2019-06-13","cited_by_count":320},{"id":"2002.09277","title":"Kernel and Rich Regimes in Overparametrized Models","created":"2020-02-20","cited_by_count":306},{"id":"1705.08741","title":"Train longer, generalize better: closing the generalization gap in large batch training of neural networks","created":"2017-05-24","cited_by_count":279},{"id":"1806.00468","title":"Implicit Bias of Gradient Descent on Linear Convolutional Networks","created":"2018-06-01","cited_by_count":251},{"id":"1702.03849","title":"Non-convex learning via Stochastic Gradient Langevin Dynamics: a nonasymptotic analysis","created":"2017-02-13","cited_by_count":243},{"id":"2205.10343","title":"Towards Understanding Grokking: An Effective Theory of Representation Learning","created":"2022-05-20","cited_by_count":230},{"id":"1808.00387","title":"Just Interpolate: Kernel \"Ridgeless\" Regression Can Generalize","created":"2018-08-01","cited_by_count":190},{"id":"2101.12176","title":"On the Origin of Implicit Regularization in Stochastic Gradient Descent","created":"2021-01-28","cited_by_count":184},{"id":"1907.04595","title":"Towards Explaining the Regularization Effect of Initial Large Learning Rate in Training Neural Networks","created":"2019-07-10","cited_by_count":162},{"id":"2209.15594","title":"Self-Stabilization: The Implicit Bias of Gradient Descent at the Edge of Stability","created":"2022-09-30","cited_by_count":155},{"id":"1511.06251","title":"Stochastic modified equations and adaptive stochastic gradient algorithms","created":"2015-11-19","cited_by_count":142},{"id":"1710.06451","title":"A Bayesian Perspective on Generalization and Stochastic Gradient Descent","created":"2017-10-17","cited_by_count":135},{"id":"2205.09745","title":"Understanding Gradient Descent on Edge of Stability in Deep Learning","created":"2022-05-19","cited_by_count":128},{"id":"2010.05627","title":"Towards Theoretically Understanding Why SGD Generalizes Better Than ADAM in Deep Learning","created":"2020-10-12","cited_by_count":126},{"id":"1904.09080","title":"Implicit regularization for deep neural networks driven by an Ornstein-Uhlenbeck like process","created":"2019-04-19","cited_by_count":123},{"id":"2106.06530","title":"Label Noise SGD Provably Prefers Flat Global Minimizers","created":"2021-06-11","cited_by_count":121},{"id":"1802.06175","title":"An Alternative View: When Does SGD Escape Local Minima?","created":"2018-02-17","cited_by_count":115},{"id":"1811.01558","title":"Stochastic Modified Equations and Dynamics of Stochastic Gradient Algorithms I: Mathematical Foundations","created":"2018-11-05","cited_by_count":114},{"id":"2106.09524","title":"Implicit Bias of SGD for Diagonal Linear Networks: a Provable Benefit of Stochasticity","created":"2021-06-17","cited_by_count":110},{"id":"2110.06914","title":"What Happens after SGD Reaches Zero Loss? --A Mathematical Framework","created":"2021-10-13","cited_by_count":107},{"id":"2006.05800","title":"On the Optimal Weighted $\\ell_2$ Regularization in Overparameterized Linear Regression","created":"2020-06-10","cited_by_count":98},{"id":"2006.08680","title":"Shape Matters: Understanding the Implicit Bias of the Noise Covariance","created":"2020-06-15","cited_by_count":88},{"id":"1909.05122","title":"Implicit Regularization for Optimal Sparse Recovery","created":"2019-09-11","cited_by_count":80},{"id":"2102.12470","title":"On the Validity of Modeling SGD with Stochastic Differential Equations (SDEs)","created":"2021-02-24","cited_by_count":73},{"id":"2310.04415","title":"Why Do We Need Weight Decay in Modern Deep Learning?","created":"2023-10-06","cited_by_count":72},{"id":"1702.05575","title":"A Hitting Time Analysis of Stochastic Gradient Langevin Dynamics","created":"2017-02-18","cited_by_count":71},{"id":"2210.05337","title":"SGD with Large Step Sizes Learns Sparse Features","created":"2022-10-11","cited_by_count":68},{"id":"2207.02628","title":"The alignment property of SGD noise and how it helps select flat minima: A stability analysis","created":"2022-07-06","cited_by_count":64},{"id":"2010.02916","title":"Reconciling Modern Deep Learning with Traditional Optimization Analyses: The Intrinsic Learning Rate","created":"2020-10-06","cited_by_count":59},{"id":"1602.02666","title":"A Variational Analysis of Stochastic Gradient Algorithms","created":"2016-02-08","cited_by_count":57},{"id":"2210.03294","title":"Understanding Edge-of-Stability Training Dynamics with a Minimalist Example","created":"2022-10-07","cited_by_count":56},{"id":"2110.03677","title":"Large Learning Rate Tames Homogeneity: Convergence and Balancing Effect","created":"2021-10-07","cited_by_count":50},{"id":"2204.11326","title":"Beyond the Quadratic Approximation: the Multiscale Structure of Neural Network Loss Landscapes","created":"2022-04-24","cited_by_count":43},{"id":"2207.05931","title":"Towards understanding how momentum improves generalization in deep learning","created":"2022-07-13","cited_by_count":40},{"id":"2105.09557","title":"Power-law escape rate of SGD","created":"2021-05-20","cited_by_count":29},{"id":"2210.15435","title":"Grokking phase transitions in learning local rules with gradient descent","created":"2022-10-26","cited_by_count":28},{"id":"1911.07956","title":"Implicit Regularization and Convergence for Weight Normalization","created":"2019-11-18","cited_by_count":20},{"id":"2307.06324","title":"Provably Faster Gradient Descent via Long Steps","created":"2023-07-12","cited_by_count":20},{"id":"1902.01028","title":"Can SGD Learn Recurrent Neural Networks with Provable Generalization?","created":"2019-02-04","cited_by_count":17},{"id":"1904.13016","title":"On Stationary-Point Hitting Time and Ergodicity of Stochastic Gradient Langevin Dynamics","created":"2019-04-30","cited_by_count":14},{"id":"2103.01338","title":"Acceleration via Fractal Learning Rate Schedules","created":"2021-03-01","cited_by_count":13},{"id":"2305.05448","title":"Robust Implicit Regularization via Weight Normalization","created":"2023-05-09","cited_by_count":12},{"id":"2202.13733","title":"On the Benefits of Large Learning Rates for Kernel Methods","created":"2022-02-28","cited_by_count":8}],"next_cursor":null}