{"list":"other","total":63,"rows":[{"id":"1512.03385","title":"Deep Residual Learning for Image Recognition","created":"2015-12-10","cited_by_count":52096},{"id":"1412.6980","title":"Adam: A Method for Stochastic Optimization","created":"2014-12-22","cited_by_count":37240},{"id":"1409.1556","title":"Very Deep Convolutional Networks for Large-Scale Image Recognition","created":"2014-09-04","cited_by_count":15364},{"id":"1608.03983","title":"SGDR: Stochastic Gradient Descent with Warm Restarts","created":"2016-08-13","cited_by_count":5045},{"id":"1806.07572","title":"Neural Tangent Kernel: Convergence and Generalization in Neural Networks","created":"2018-06-20","cited_by_count":2621},{"id":"2010.01412","title":"Sharpness-Aware Minimization for Efficiently Improving Generalization","created":"2020-10-03","cited_by_count":1417},{"id":"1609.04836","title":"On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima","created":"2016-09-15","cited_by_count":1352},{"id":"1712.09913","title":"Visualizing the Loss Landscape of Neural Nets","created":"2017-12-28","cited_by_count":1130},{"id":"1511.07289","title":"Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)","created":"2015-11-23","cited_by_count":1070},{"id":"1503.05671","title":"Optimizing Neural Networks with Kronecker-factored Approximate Curvature","created":"2015-03-19","cited_by_count":796},{"id":"1811.03962","title":"A Convergence Theory for Deep Learning via Over-Parameterization","created":"2018-11-09","cited_by_count":787},{"id":"1312.6120","title":"Exact solutions to the nonlinear dynamics of learning in deep linear neural networks","created":"2013-12-20","cited_by_count":763},{"id":"1902.06720","title":"Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent","created":"2019-02-18","cited_by_count":738},{"id":"1811.03804","title":"Gradient Descent Finds Global Minima of Deep Neural Networks","created":"2018-11-09","cited_by_count":574},{"id":"1812.01187","title":"Bag of Tricks for Image Classification with Convolutional Neural Networks","created":"2018-12-04","cited_by_count":431},{"id":"2103.00065","title":"Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability","created":"2021-02-26","cited_by_count":427},{"id":"1901.10159","title":"An Investigation into Neural Net Optimization via Hessian Eigenvalue Density","created":"2019-01-29","cited_by_count":292},{"id":"1706.04454","title":"Empirical Analysis of the Hessian of Over-Parametrized Neural Networks","created":"2017-06-14","cited_by_count":290},{"id":"1406.2572","title":"Identifying and attacking the saddle point problem in high-dimensional non-convex optimization","created":"2014-06-10","cited_by_count":289},{"id":"1802.06509","title":"On the Optimization of Deep Networks: Implicit Acceleration by Overparameterization","created":"2018-02-19","cited_by_count":284},{"id":"1806.00468","title":"Implicit Bias of Gradient Descent on Linear Convolutional Networks","created":"2018-06-01","cited_by_count":251},{"id":"1710.03667","title":"High-dimensional dynamics of generalization error in neural networks","created":"2017-10-10","cited_by_count":241},{"id":"1605.07110","title":"Deep Learning without Poor Local Minima","created":"2016-05-23","cited_by_count":234},{"id":"1812.04754","title":"Gradient Descent Happens in a Tiny Subspace","created":"2018-12-12","cited_by_count":217},{"id":"2003.02218","title":"The large learning rate phase of deep learning: the catapult mechanism","created":"2020-03-04","cited_by_count":216},{"id":"2101.12176","title":"On the Origin of Implicit Regularization in Stochastic Gradient Descent","created":"2021-01-28","cited_by_count":184},{"id":"1806.00900","title":"Algorithmic Regularization in Learning Deep Homogeneous Models: Layers are Automatically Balanced","created":"2018-06-04","cited_by_count":177},{"id":"1907.04595","title":"Towards Explaining the Regularization Effect of Initial Large Learning Rate in Training Neural Networks","created":"2019-07-10","cited_by_count":162},{"id":"2007.15801","title":"Finite Versus Infinite Neural Networks: an Empirical Study","created":"2020-07-31","cited_by_count":157},{"id":"2209.15594","title":"Self-Stabilization: The Implicit Bias of Gradient Descent at the Edge of Stability","created":"2022-09-30","cited_by_count":155},{"id":"2002.09572","title":"The Break-Even Point on Optimization Trajectories of Deep Neural Networks","created":"2020-02-21","cited_by_count":150},{"id":"2009.11162","title":"Implicit Gradient Regularization","created":"2020-09-23","cited_by_count":146},{"id":"1511.06251","title":"Stochastic modified equations and adaptive stochastic gradient algorithms","created":"2015-11-19","cited_by_count":142},{"id":"2205.09745","title":"Understanding Gradient Descent on Edge of Stability in Deep Learning","created":"2022-05-19","cited_by_count":128},{"id":"1802.05642","title":"The Mechanics of n-Player Differentiable Games","created":"2018-02-15","cited_by_count":118},{"id":"1705.10461","title":"The Numerics of GANs","created":"2017-05-30","cited_by_count":105},{"id":"1807.05031","title":"On the Relation Between the Sharpest Directions of DNN Loss and the SGD Step Length","created":"2018-07-13","cited_by_count":88},{"id":"2204.01050","title":"Understanding the unstable convergence of gradient descent","created":"2022-04-03","cited_by_count":86},{"id":"1809.10374","title":"An analytic theory of generalization dynamics and transfer learning in deep linear networks","created":"2018-09-27","cited_by_count":85},{"id":"1705.10412","title":"Gradient Descent Can Take Exponential Time to Escape Saddle Points","created":"2017-05-29","cited_by_count":74},{"id":"1810.08907","title":"Understanding the Acceleration Phenomenon via High-Resolution Differential Equations","created":"2018-10-21","cited_by_count":69},{"id":"1706.04156","title":"Gradient descent GAN optimization is locally stable","created":"2017-06-13","cited_by_count":67},{"id":"2109.14119","title":"Stochastic Training is Not Necessary for Generalization","created":"2021-09-29","cited_by_count":63},{"id":"2206.10654","title":"On the Maximum Hessian Eigenvalue and Generalization","created":"2022-06-21","cited_by_count":57},{"id":"1901.08572","title":"Width Provably Matters in Optimization for Deep Linear Neural Networks","created":"2019-01-24","cited_by_count":52},{"id":"1810.00004","title":"Fluctuation-dissipation relations for stochastic gradient descent","created":"2018-09-28","cited_by_count":48},{"id":"2012.05156","title":"Implicit Regularization in ReLU Networks with the Square Loss","created":"2020-12-09","cited_by_count":45},{"id":"1802.06093","title":"Gradient descent with identity initialization efficiently learns positive definite linear transformations by deep residual networks","created":"2018-02-16","cited_by_count":44},{"id":"2110.04369","title":"A Loss Curvature Perspective on Training Instability in Deep Learning","created":"2021-10-08","cited_by_count":40},{"id":"2102.05375","title":"Strength of Minibatch Noise in SGD","created":"2021-02-10","cited_by_count":39},{"id":"2012.03636","title":"Noise and Fluctuation of Finite Learning Rate Stochastic Gradient Descent","created":"2020-12-07","cited_by_count":37},{"id":"2107.06608","title":"Continuous vs. Discrete Optimization of Deep Neural Networks","created":"2021-07-14","cited_by_count":34},{"id":"1906.04285","title":"Continuous Time Analysis of Momentum Methods","created":"2019-06-10","cited_by_count":32},{"id":"1303.0818","title":"Riemannian metrics for neural networks I: feedforward networks","created":"2013-03-04","cited_by_count":30},{"id":"1903.04100","title":"Conformal Symplectic and Relativistic Optimization","created":"2019-03-11","cited_by_count":29},{"id":"2004.06840","title":"On dissipative symplectic integration with applications to gradient-based optimization","created":"2020-04-15","cited_by_count":23},{"id":"2010.15040","title":"Training Generative Adversarial Networks by Solving Ordinary Differential Equations","created":"2020-10-28","cited_by_count":21},{"id":"1803.01273","title":"Accelerating Natural Gradient with Higher-Order Invariance","created":"2018-03-04","cited_by_count":17},{"id":"1804.05012","title":"Representing smooth functions as compositions of near-identity functions with implications for deep network optimization","created":"2018-04-13","cited_by_count":15},{"id":"2105.13922","title":"Discretization Drift in Two-Player Games","created":"2021-05-28","cited_by_count":14},{"id":"2210.15898","title":"Toward Equation of Motion for Deep Neural Networks: Continuous-time Gradient Descent and Discretization Error Analysis","created":"2022-10-28","cited_by_count":13},{"id":"2112.13826","title":"Last-Iterate Convergence of Saddle-Point Optimizers via High-Resolution Differential Equations","created":"2021-12-27","cited_by_count":8},{"id":"1306.0514","title":"Riemannian metrics for neural networks II: recurrent networks and learning symbolic data sequences","created":"2013-06-03","cited_by_count":2}],"next_cursor":null}