{"list":"other","total":51,"rows":[{"id":"1811.03962","title":"A Convergence Theory for Deep Learning via Over-Parameterization","created":"2018-11-09","cited_by_count":787},{"id":"1312.6120","title":"Exact solutions to the nonlinear dynamics of learning in deep linear neural networks","created":"2013-12-20","cited_by_count":763},{"id":"1811.03804","title":"Gradient Descent Finds Global Minima of Deep Neural Networks","created":"2018-11-09","cited_by_count":574},{"id":"2103.00065","title":"Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability","created":"2021-02-26","cited_by_count":427},{"id":"1905.13655","title":"Implicit Regularization in Deep Matrix Factorization","created":"2019-05-31","cited_by_count":405},{"id":"1906.05890","title":"Gradient Descent Maximizes the Margin of Homogeneous Neural Networks","created":"2019-06-13","cited_by_count":320},{"id":"2002.09277","title":"Kernel and Rich Regimes in Overparametrized Models","created":"2020-02-20","cited_by_count":306},{"id":"1802.06509","title":"On the Optimization of Deep Networks: Implicit Acceleration by Overparameterization","created":"2018-02-19","cited_by_count":284},{"id":"2002.04486","title":"Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Loss","created":"2020-02-11","cited_by_count":268},{"id":"1806.00468","title":"Implicit Bias of Gradient Descent on Linear Convolutional Networks","created":"2018-06-01","cited_by_count":251},{"id":"1702.03849","title":"Non-convex learning via Stochastic Gradient Langevin Dynamics: a nonasymptotic analysis","created":"2017-02-13","cited_by_count":243},{"id":"1710.03667","title":"High-dimensional dynamics of generalization error in neural networks","created":"2017-10-10","cited_by_count":241},{"id":"2003.02218","title":"The large learning rate phase of deep learning: the catapult mechanism","created":"2020-03-04","cited_by_count":216},{"id":"1503.01243","title":"A Differential Equation for Modeling Nesterov's Accelerated Gradient Method: Theory and Insights","created":"2015-03-04","cited_by_count":206},{"id":"1810.02281","title":"A Convergence Analysis of Gradient Descent for Deep Linear Neural Networks","created":"2018-10-04","cited_by_count":184},{"id":"2101.12176","title":"On the Origin of Implicit Regularization in Stochastic Gradient Descent","created":"2021-01-28","cited_by_count":184},{"id":"1810.02032","title":"Gradient descent aligns the layers of deep linear networks","created":"2018-10-04","cited_by_count":182},{"id":"2006.06657","title":"Directional convergence and alignment in deep learning","created":"2020-06-11","cited_by_count":178},{"id":"1806.00900","title":"Algorithmic Regularization in Learning Deep Homogeneous Models: Layers are Automatically Balanced","created":"2018-06-04","cited_by_count":177},{"id":"1907.04595","title":"Towards Explaining the Regularization Effect of Initial Large Learning Rate in Training Neural Networks","created":"2019-07-10","cited_by_count":162},{"id":"2002.09572","title":"The Break-Even Point on Optimization Trajectories of Deep Neural Networks","created":"2020-02-21","cited_by_count":150},{"id":"2009.11162","title":"Implicit Gradient Regularization","created":"2020-09-23","cited_by_count":146},{"id":"1511.06251","title":"Stochastic modified equations and adaptive stochastic gradient algorithms","created":"2015-11-19","cited_by_count":142},{"id":"1811.08888","title":"Stochastic Gradient Descent Optimizes Over-parameterized Deep ReLU Networks","created":"2018-11-21","cited_by_count":141},{"id":"1804.07795","title":"Stochastic subgradient method converges on tame functions","created":"2018-04-20","cited_by_count":124},{"id":"1906.04688","title":"An Improved Analysis of Training Over-parameterized Deep Neural Networks","created":"2019-06-11","cited_by_count":121},{"id":"2005.06398","title":"Implicit Regularization in Deep Learning May Not Be Explainable by Norms","created":"2020-05-13","cited_by_count":119},{"id":"1603.04245","title":"A Variational Perspective on Accelerated Methods in Optimization","created":"2016-03-14","cited_by_count":110},{"id":"2012.04728","title":"Neural Mechanics: Symmetry and Broken Conservation Laws in Deep Learning Dynamics","created":"2020-12-08","cited_by_count":96},{"id":"1809.10374","title":"An analytic theory of generalization dynamics and transfer learning in deep linear networks","created":"2018-09-27","cited_by_count":85},{"id":"2010.02501","title":"A Unifying View on Implicit Bias in Training Linear Neural Networks","created":"2020-10-06","cited_by_count":75},{"id":"2001.05992","title":"Provable Benefit of Orthogonal Initialization in Optimizing Deep Linear Networks","created":"2020-01-16","cited_by_count":73},{"id":"2102.09769","title":"On the Implicit Bias of Initialization Shape: Beyond Infinitesimal Mirror Descent","created":"2021-02-19","cited_by_count":70},{"id":"1810.08907","title":"Understanding the Acceleration Phenomenon via High-Resolution Differential Equations","created":"2018-10-21","cited_by_count":69},{"id":"1901.08572","title":"Width Provably Matters in Optimization for Deep Linear Neural Networks","created":"2019-01-24","cited_by_count":52},{"id":"2012.05156","title":"Implicit Regularization in ReLU Networks with the Square Loss","created":"2020-12-09","cited_by_count":45},{"id":"1802.06093","title":"Gradient descent with identity initialization efficiently learns positive definite linear transformations by deep residual networks","created":"2018-02-16","cited_by_count":44},{"id":"2102.09972","title":"Implicit Regularization in Tensor Factorization","created":"2021-02-19","cited_by_count":44},{"id":"2011.13772","title":"Gradient Descent for Deep Matrix Factorization: Dynamics and Implicit Bias towards Low Rank","created":"2020-11-27","cited_by_count":42},{"id":"2003.01094","title":"On the Global Convergence of Training Deep Linear ResNets","created":"2020-03-02","cited_by_count":29},{"id":"1611.02635","title":"A Lyapunov Analysis of Momentum Methods in Optimization","created":"2016-11-08","cited_by_count":28},{"id":"1809.08587","title":"Exponential Convergence Time of Gradient Descent for One-Dimensional Deep Linear Neural Networks","created":"2018-09-23","cited_by_count":22},{"id":"1805.00521","title":"Direct Runge-Kutta Discretization Achieves Acceleration","created":"2018-05-01","cited_by_count":21},{"id":"2002.09852","title":"Training Linear Neural Networks: Non-Local Convergence and Complexity Results","created":"2020-02-23","cited_by_count":19},{"id":"1911.00645","title":"Global Convergence of Gradient Descent for Deep Linear Residual Networks","created":"2019-11-02","cited_by_count":16},{"id":"1904.05263","title":"Analysis of the Gradient Descent Algorithm for a Deep Neural Network Model with Skip-connections","created":"2019-04-10","cited_by_count":7},{"id":"1702.06751","title":"Integration Methods and Accelerated Optimization Algorithms","created":"2017-02-22","cited_by_count":6},{"id":"1805.06579","title":"ADMM and Accelerated ADMM as Continuous Dynamical Systems","created":"2018-05-17","cited_by_count":6},{"id":"1911.05206","title":"Shadowing Properties of Optimization Algorithms","created":"2019-11-12","cited_by_count":6},{"id":"2101.04243","title":"A Convergence Theory Towards Practical Over-parameterized Deep Neural Networks","created":"2021-01-12","cited_by_count":4},{"id":"2004.08981","title":"Stochastic gradient algorithms from ODE splitting perspective","created":"2020-04-19","cited_by_count":2}],"next_cursor":null}