{"list":"other","total":53,"rows":[{"id":"1512.03385","title":"Deep Residual Learning for Image Recognition","created":"2015-12-10","cited_by_count":52096},{"id":"2010.11929","title":"An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale","created":"2020-10-22","cited_by_count":30186},{"id":"1502.03167","title":"Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift","created":"2015-02-11","cited_by_count":7005},{"id":"1607.06450","title":"Layer Normalization","created":"2016-07-21","cited_by_count":5669},{"id":"1502.01852","title":"Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification","created":"2015-02-06","cited_by_count":3611},{"id":"1605.07146","title":"Wide Residual Networks","created":"2016-05-23","cited_by_count":3070},{"id":"1909.13719","title":"RandAugment: Practical automated data augmentation with a reduced search space","created":"2019-09-30","cited_by_count":2269},{"id":"1803.05407","title":"Averaging Weights Leads to Wider Optima and Better Generalization","created":"2018-03-14","cited_by_count":1209},{"id":"1710.03740","title":"Mixed Precision Training","created":"2017-10-10","cited_by_count":1156},{"id":"2008.03703","title":"What Neural Networks Memorize and Why: Discovering the Long Tail via Influence Estimation","created":"2020-08-09","cited_by_count":497},{"id":"2103.00065","title":"Gradient Descent on Neural Networks Typically Occurs at the Edge of Stability","created":"2021-02-26","cited_by_count":427},{"id":"2204.02937","title":"Last Layer Re-Training is Sufficient for Robustness to Spurious Correlations","created":"2022-04-06","cited_by_count":381},{"id":"1906.05890","title":"Gradient Descent Maximizes the Margin of Homogeneous Neural Networks","created":"2019-06-13","cited_by_count":320},{"id":"2202.06709","title":"How Do Vision Transformers Work?","created":"2022-02-14","cited_by_count":307},{"id":"2012.09816","title":"Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learning","created":"2020-12-17","cited_by_count":297},{"id":"2110.06296","title":"The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networks","created":"2021-10-12","cited_by_count":291},{"id":"1906.08988","title":"A Fourier Perspective on Model Robustness in Computer Vision","created":"2019-06-21","cited_by_count":288},{"id":"1711.04623","title":"Three Factors Influencing Minima in SGD","created":"2017-11-13","cited_by_count":273},{"id":"2003.02218","title":"The large learning rate phase of deep learning: the catapult mechanism","created":"2020-03-04","cited_by_count":216},{"id":"2101.12176","title":"On the Origin of Implicit Regularization in Stochastic Gradient Descent","created":"2021-01-28","cited_by_count":184},{"id":"1907.04595","title":"Towards Explaining the Regularization Effect of Initial Large Learning Rate in Training Neural Networks","created":"2019-07-10","cited_by_count":162},{"id":"2002.09572","title":"The Break-Even Point on Optimization Trajectories of Deep Neural Networks","created":"2020-02-21","cited_by_count":150},{"id":"2009.11162","title":"Implicit Gradient Regularization","created":"2020-09-23","cited_by_count":146},{"id":"1802.06175","title":"An Alternative View: When Does SGD Escape Local Minima?","created":"2018-02-17","cited_by_count":115},{"id":"2302.07011","title":"A Modern Look at the Relationship between Sharpness and Generalization","created":"2023-02-14","cited_by_count":92},{"id":"1807.05031","title":"On the Relation Between the Sharpest Directions of DNN Loss and the SGD Step Length","created":"2018-07-13","cited_by_count":88},{"id":"1802.08770","title":"A Walk with SGD","created":"2018-02-24","cited_by_count":78},{"id":"2310.04415","title":"Why Do We Need Weight Decay in Modern Deep Learning?","created":"2023-10-06","cited_by_count":72},{"id":"2210.05337","title":"SGD with Large Step Sizes Learns Sparse Features","created":"2022-10-11","cited_by_count":68},{"id":"1812.03981","title":"Theoretical Analysis of Auto Rate-Tuning by Batch Normalization","created":"2018-12-10","cited_by_count":65},{"id":"2207.02628","title":"The alignment property of SGD noise and how it helps select flat minima: A stability analysis","created":"2022-07-06","cited_by_count":64},{"id":"2105.13462","title":"On Linear Stability of SGD and Input-Smoothness of Neural Networks","created":"2021-05-27","cited_by_count":63},{"id":"2012.14193","title":"Catastrophic Fisher Explosion: Early Phase Fisher Matrix Impacts Generalization","created":"2020-12-28","cited_by_count":62},{"id":"2010.02916","title":"Reconciling Modern Deep Learning with Traditional Optimization Analyses: The Intrinsic Learning Rate","created":"2020-10-06","cited_by_count":59},{"id":"2206.10654","title":"On the Maximum Hessian Eigenvalue and Generalization","created":"2022-06-21","cited_by_count":57},{"id":"2212.07469","title":"Learning threshold neurons via the \"edge of stability\"","created":"2022-12-14","cited_by_count":51},{"id":"2110.03677","title":"Large Learning Rate Tames Homogeneity: Convergence and Balancing Effect","created":"2021-10-07","cited_by_count":50},{"id":"2305.17490","title":"The Implicit Regularization of Dynamical Stability in Stochastic Gradient Descent","created":"2023-05-27","cited_by_count":44},{"id":"2306.04251","title":"Stochastic Collapse: How Gradient Noise Attracts SGD Dynamics Towards Simpler Subnetworks","created":"2023-06-07","cited_by_count":39},{"id":"2306.06179","title":"Hidden symmetries of ReLU networks","created":"2023-06-09","cited_by_count":39},{"id":"2004.06977","title":"On Learning Rates and Schr\\\"odinger Operators","created":"2020-04-15","cited_by_count":33},{"id":"2206.13424","title":"Benchopt: Reproducible, efficient and collaborative optimization benchmarks","created":"2022-06-27","cited_by_count":27},{"id":"2003.03977","title":"Wide-minima Density Hypothesis and the Explore-Exploit Learning Rate Schedule","created":"2020-03-09","cited_by_count":25},{"id":"2302.00849","title":"Implicit regularization in Heavy-ball momentum accelerated stochastic gradient descent","created":"2023-02-02","cited_by_count":25},{"id":"2206.01246","title":"Stochastic gradient descent introduces an effective landscape-dependent regularization favoring flat solutions","created":"2022-06-02","cited_by_count":22},{"id":"2106.15739","title":"On the Periodic Behavior of Neural Network Training with Batch Normalization and Weight Decay","created":"2021-06-29","cited_by_count":20},{"id":"2209.03695","title":"Training Scale-Invariant Neural Networks on the Sphere Can Happen in Three Regimes","created":"2022-09-08","cited_by_count":19},{"id":"2306.17499","title":"The Implicit Bias of Minima Stability in Multivariate Shallow ReLU Networks","created":"2023-06-30","cited_by_count":15},{"id":"2311.04163","title":"Outliers with Opposing Signals Have an Outsized Effect on Neural Network Optimization","created":"2023-11-07","cited_by_count":13},{"id":"2406.06838","title":"Stable Minima Cannot Overfit in Univariate ReLU Networks: Generalization by Large Step Sizes","created":"2024-06-10","cited_by_count":12},{"id":"2205.15142","title":"Special Properties of Gradient Descent with Large Learning Rates","created":"2022-05-30","cited_by_count":10},{"id":"2310.17074","title":"Benign Oscillation of Stochastic Gradient Descent with Large Learning Rates","created":"2023-10-26","cited_by_count":8},{"id":"2401.11600","title":"Understanding the Generalization Benefits of Late Learning Rate Decay","created":"2024-01-21","cited_by_count":6}],"next_cursor":null}