{"list":"other","total":99,"rows":[{"id":"1512.03385","title":"Deep Residual Learning for Image Recognition","created":"2015-12-10","cited_by_count":52096},{"id":"1810.04805","title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding","created":"2018-10-11","cited_by_count":40399},{"id":"2005.14165","title":"Language Models are Few-Shot Learners","created":"2020-05-28","cited_by_count":32419},{"id":"1409.1556","title":"Very Deep Convolutional Networks for Large-Scale Image Recognition","created":"2014-09-04","cited_by_count":15364},{"id":"1502.03167","title":"Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift","created":"2015-02-11","cited_by_count":7005},{"id":"1607.06450","title":"Layer Normalization","created":"2016-07-21","cited_by_count":5669},{"id":"1512.00567","title":"Rethinking the Inception Architecture for Computer Vision","created":"2015-12-02","cited_by_count":5475},{"id":"1606.08415","title":"Gaussian Error Linear Units (GELUs)","created":"2016-06-27","cited_by_count":3502},{"id":"1806.07572","title":"Neural Tangent Kernel: Convergence and Generalization in Neural Networks","created":"2018-06-20","cited_by_count":2621},{"id":"1603.05027","title":"Identity Mappings in Deep Residual Networks","created":"2016-03-16","cited_by_count":1965},{"id":"1611.03530","title":"Understanding deep learning requires rethinking generalization","created":"2016-11-10","cited_by_count":1674},{"id":"1803.08494","title":"Group Normalization","created":"2018-03-22","cited_by_count":1524},{"id":"2010.01412","title":"Sharpness-Aware Minimization for Efficiently Improving Generalization","created":"2020-10-03","cited_by_count":1417},{"id":"1609.04836","title":"On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima","created":"2016-09-15","cited_by_count":1352},{"id":"1712.09913","title":"Visualizing the Loss Landscape of Neural Nets","created":"2017-12-28","cited_by_count":1130},{"id":"1811.03962","title":"A Convergence Theory for Deep Learning via Over-Parameterization","created":"2018-11-09","cited_by_count":787},{"id":"1812.07956","title":"On Lazy Training in Differentiable Programming","created":"2018-12-19","cited_by_count":675},{"id":"1710.10345","title":"The Implicit Bias of Gradient Descent on Separable Data","created":"2017-10-27","cited_by_count":655},{"id":"1706.08947","title":"Exploring Generalization in Deep Learning","created":"2017-06-27","cited_by_count":600},{"id":"1904.11955","title":"On Exact Computation with an Infinitely Wide Neural Net","created":"2019-04-26","cited_by_count":573},{"id":"1901.08584","title":"Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks","created":"2019-01-24","cited_by_count":566},{"id":"1602.07868","title":"Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks","created":"2016-02-25","cited_by_count":543},{"id":"1608.04636","title":"Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-\\L{}ojasiewicz Condition","created":"2016-08-16","cited_by_count":539},{"id":"1703.04933","title":"Sharp Minima Can Generalize For Deep Nets","created":"2017-03-15","cited_by_count":466},{"id":"2103.00065","title":"Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability","created":"2021-02-26","cited_by_count":427},{"id":"1905.13655","title":"Implicit Regularization in Deep Matrix Factorization","created":"2019-05-31","cited_by_count":405},{"id":"1805.11604","title":"How Does Batch Normalization Help Optimization?","created":"2018-05-29","cited_by_count":357},{"id":"1811.04918","title":"Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layers","created":"2018-11-12","cited_by_count":340},{"id":"1705.09280","title":"Implicit Regularization in Matrix Factorization","created":"2017-05-25","cited_by_count":322},{"id":"1906.05890","title":"Gradient Descent Maximizes the Margin of Homogeneous Neural Networks","created":"2019-06-13","cited_by_count":320},{"id":"1901.10159","title":"An Investigation into Neural Net Optimization via Hessian Eigenvalue Density","created":"2019-01-29","cited_by_count":292},{"id":"1705.08741","title":"Train longer, generalize better: closing the generalization gap in large batch training of neural networks","created":"2017-05-24","cited_by_count":279},{"id":"2002.04486","title":"Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Loss","created":"2020-02-11","cited_by_count":268},{"id":"1808.01204","title":"Learning Overparameterized Neural Networks via Stochastic Gradient Descent on Structured Data","created":"2018-08-03","cited_by_count":261},{"id":"1806.00468","title":"Implicit Bias of Gradient Descent on Linear Convolutional Networks","created":"2018-06-01","cited_by_count":251},{"id":"1905.13210","title":"Generalization Bounds of Stochastic Gradient Descent for Wide and Deep Neural Networks","created":"2019-05-30","cited_by_count":222},{"id":"2003.02218","title":"The large learning rate phase of deep learning: the catapult mechanism","created":"2020-03-04","cited_by_count":216},{"id":"2006.06657","title":"Directional convergence and alignment in deep learning","created":"2020-06-11","cited_by_count":178},{"id":"1809.09573","title":"Nonconvex Optimization Meets Low-Rank Matrix Factorization: An Overview","created":"2018-09-25","cited_by_count":160},{"id":"1806.02375","title":"Understanding Batch Normalization","created":"2018-06-01","cited_by_count":159},{"id":"1702.08591","title":"The Shattered Gradients Problem: If resnets are the answer, then what is the question?","created":"2017-02-28","cited_by_count":157},{"id":"2002.09572","title":"The Break-Even Point on Optimization Trajectories of Deep Neural Networks","created":"2020-02-21","cited_by_count":150},{"id":"2012.09839","title":"Towards Resolving the Implicit Bias of Gradient Descent for Matrix Factorization: Greedy Low-Rank Learning","created":"2020-12-17","cited_by_count":150},{"id":"1902.04760","title":"Scaling Limits of Wide Neural Networks with Weight Sharing: Gaussian Process Behavior, Gradient Independence, and Neural Tangent Kernel Derivation","created":"2019-02-13","cited_by_count":149},{"id":"2009.11162","title":"Implicit Gradient Regularization","created":"2020-09-23","cited_by_count":146},{"id":"1811.08888","title":"Stochastic Gradient Descent Optimizes Over-parameterized Deep ReLU Networks","created":"2018-11-21","cited_by_count":141},{"id":"2205.09745","title":"Understanding Gradient Descent on Edge of Stability in Deep Learning","created":"2022-05-19","cited_by_count":128},{"id":"1706.05350","title":"L2 Regularization versus Batch and Weight Normalization","created":"2017-06-16","cited_by_count":124},{"id":"1904.09080","title":"Implicit regularization for deep neural networks driven by an Ornstein-Uhlenbeck like process","created":"2019-04-19","cited_by_count":123},{"id":"1910.07454","title":"An Exponential Learning Rate Schedule for Deep Learning","created":"2019-10-16","cited_by_count":121},{"id":"2106.06530","title":"Label Noise SGD Provably Prefers Flat Global Minimizers","created":"2021-06-11","cited_by_count":121},{"id":"2005.06398","title":"Implicit Regularization in Deep Learning May Not Be Explainable by Norms","created":"2020-05-13","cited_by_count":119},{"id":"1802.06175","title":"An Alternative View: When Does SGD Escape Local Minima?","created":"2018-02-17","cited_by_count":115},{"id":"1810.12281","title":"Three Mechanisms of Weight Decay Regularization","created":"2018-10-29","cited_by_count":113},{"id":"2110.06914","title":"What Happens after SGD Reaches Zero Loss? --A Mathematical Framework","created":"2021-10-13","cited_by_count":107},{"id":"1902.00744","title":"Asymmetric Valleys: Beyond Sharp and Flat Local Minima","created":"2019-02-02","cited_by_count":90},{"id":"2006.08680","title":"Shape Matters: Understanding the Implicit Bias of the Noise Covariance","created":"2020-06-15","cited_by_count":88},{"id":"2101.08692","title":"Characterizing signal propagation to close the performance gap in unnormalized ResNets","created":"2021-01-21","cited_by_count":88},{"id":"2204.01050","title":"Understanding the unstable convergence of gradient descent","created":"2022-04-03","cited_by_count":86},{"id":"1706.10239","title":"Towards Understanding Generalization of Deep Learning: Perspective of Loss Landscapes","created":"2017-06-30","cited_by_count":84},{"id":"1911.12360","title":"How Much Over-parameterization Is Sufficient to Learn Deep ReLU Networks?","created":"2019-11-27","cited_by_count":84},{"id":"2110.13905","title":"Gradient Descent on Two-layer Nets: Margin Maximization and Simplicity Bias","created":"2021-10-26","cited_by_count":83},{"id":"2106.15013","title":"Small random initialization is akin to spectral learning: Optimization and generalization guarantees for overparameterized low-rank matrix reconstruction","created":"2021-06-28","cited_by_count":78},{"id":"1802.06455","title":"Bayesian Uncertainty Estimation for Batch Normalized Deep Networks","created":"2018-02-18","cited_by_count":70},{"id":"1904.01517","title":"Convergence rates for the stochastic gradient descent method for non-convex objective functions","created":"2019-04-02","cited_by_count":66},{"id":"1812.03981","title":"Theoretical Analysis of Auto Rate-Tuning by Batch Normalization","created":"2018-12-10","cited_by_count":65},{"id":"2105.13462","title":"On Linear Stability of SGD and Input-Smoothness of Neural Networks","created":"2021-05-27","cited_by_count":63},{"id":"1803.01814","title":"Norm matters: efficient and accurate normalization schemes in deep networks","created":"2018-03-05","cited_by_count":62},{"id":"1901.04653","title":"Normalized Flat Minima: Exploring Scale Invariant Definition of Flat Minima for Neural Networks using PAC-Bayesian Analysis","created":"2019-01-15","cited_by_count":59},{"id":"2010.02916","title":"Reconciling Modern Deep Learning with Traditional Optimization Analyses: The Intrinsic Learning Rate","created":"2020-10-06","cited_by_count":59},{"id":"1809.00846","title":"Towards Understanding Regularization in Batch Normalization","created":"2018-09-04","cited_by_count":56},{"id":"2105.02716","title":"Noether's Learning Dynamics: Role of Symmetry Breaking in Neural Networks","created":"2021-05-06","cited_by_count":50},{"id":"2110.03677","title":"Large Learning Rate Tames Homogeneity: Convergence and Balancing Effect","created":"2021-10-07","cited_by_count":50},{"id":"1906.02613","title":"Bad Global Minima Exist and SGD Can Reach Them","created":"2019-06-06","cited_by_count":49},{"id":"2204.11326","title":"Beyond the Quadratic Approximation: the Multiscale Structure of Neural Network Loss Landscapes","created":"2022-04-24","cited_by_count":43},{"id":"2006.08643","title":"On the training dynamics of deep networks with $L_2$ regularization","created":"2020-06-15","cited_by_count":39},{"id":"1605.00405","title":"Gradient Descent Only Converges to Minimizers: Non-Isolated Critical Points and Invariant Regions","created":"2016-05-02","cited_by_count":34},{"id":"1905.05894","title":"Online Normalization for Training Neural Networks","created":"2019-05-15","cited_by_count":29},{"id":"2106.03970","title":"Batch Normalization Orthogonalizes Representations in Deep Random Networks","created":"2021-06-07","cited_by_count":29},{"id":"2202.00980","title":"Robust Training of Neural Networks Using Scale Invariant Architectures","created":"2022-02-02","cited_by_count":29},{"id":"2203.03756","title":"Flat minima generalize for low-rank matrix recovery","created":"2022-03-07","cited_by_count":28},{"id":"1805.10694","title":"Exponential convergence rates for Batch Normalization: The power of length-direction decoupling in non-convex optimization","created":"2018-05-27","cited_by_count":27},{"id":"2201.11729","title":"Implicit Regularization in Hierarchical Tensor Factorization and Deep Convolutional Neural Networks","created":"2022-01-27","cited_by_count":27},{"id":"2002.06189","title":"Stochasticity of Deterministic Gradient Descent: Large Learning Rate for Multiscale Objective Function","created":"2020-02-14","cited_by_count":26},{"id":"2106.05956","title":"Beyond BatchNorm: Towards a Unified Understanding of Normalization in Deep Learning","created":"2021-06-10","cited_by_count":25},{"id":"2009.02773","title":"Why Spectral Normalization Stabilizes GANs: Analysis and Improvements","created":"2020-09-06","cited_by_count":21},{"id":"1906.02926","title":"The Normalization Method for Alleviating Pathological Sharpness in Wide Neural Networks","created":"2019-06-07","cited_by_count":20},{"id":"1911.07956","title":"Implicit Regularization and Convergence for Weight Normalization","created":"2019-11-18","cited_by_count":20},{"id":"2106.06573","title":"Understanding Deflation Process in Over-parametrized Tensor Decomposition","created":"2021-06-11","cited_by_count":20},{"id":"2106.15739","title":"On the Periodic Behavior of Neural Network Training with Batch Normalization and Weight Decay","created":"2021-06-29","cited_by_count":20},{"id":"2006.06878","title":"Optimization Theory for ReLU Neural Networks Trained with Normalization Layers","created":"2020-06-11","cited_by_count":15},{"id":"1710.07406","title":"First-order Methods Almost Always Avoid Saddle Points","created":"2017-10-20","cited_by_count":14},{"id":"2106.03743","title":"Proxy-Normalizing Activations to Match Batch Normalization while Removing Batch Dependence","created":"2021-06-07","cited_by_count":12},{"id":"1903.02237","title":"Positively Scale-Invariant Flatness of ReLU Neural Networks","created":"2019-03-06","cited_by_count":11},{"id":"2108.01110","title":"Batch Normalization Preconditioning for Neural Network Training","created":"2021-08-02","cited_by_count":5},{"id":"2111.04004","title":"Exponential escape efficiency of SGD from sharp minima in non-stationary regime","created":"2021-11-07","cited_by_count":5},{"id":"1811.00639","title":"Stochastic Normalizations as Bayesian Learning","created":"2018-11-01","cited_by_count":3},{"id":"2101.02944","title":"BN-invariant sharpness regularizes the training model to better generalization","created":"2021-01-08","cited_by_count":3},{"id":"2110.08725","title":"A Riemannian Mean Field Formulation for Two-layer Neural Networks with Batch Normalization","created":"2021-10-17","cited_by_count":1}],"next_cursor":null}